Making good artificial intelligence for computer opponents is difficult. Programming a game like draughts isn't terribly challenging; adding convincing AI, however, might take longer than making the game itself. But what are we doing when we make this AI? Is it important for it to examine every possible move on the board, or can we cheat and just make it move randomly? What does an AI need to do, and (most importantly) how can we ensure that the game is fun?
Good AI vs Fun AI
The term "AI" is a really broad concept, and the type of AI depends on the game. Asymmetric games, such as Space Invaders, Mario, or Tower Defence, allow the player to cut through swathes of enemies effortlessly. Since the "teams" (player vs everything else) are unbalanced, these games tend to use dumb AI. Symmetric games, such as Starcraft, Street Fighter or Civilisation, expect the player to beat opponents on equal footing, and so require more complex AI.
However, games have to be fun, first and foremost. Players enjoy a challenge, not losing repeatedly. In fact, some games such as Checkers have been "solved", meaning it is possible to build an AI which plays an unbeatable game.
When my team and I were working on our computer/board game Infected, we had to make the first level's AI intentionally awful so as not to scare players off. The game AI was initially rather simplistic, and the computer would play hyper aggressively, focusing entirely on high scoring plays with no thought of defence or board presence.
Even though the AI was poor, we found that new players would get crushed and lose interest. We revised the AI several times, although it still proved too much of a challenge for players. Eventually, we simply removed the AI, so the game would just make a random move every time. Although some players still managed to lose, this massively improved player retention, allowing us to reintroduce the "serious" AI at higher levels.
The problem of "too good" AI can be seen in a variety of games: FPS games with bots that have 360 degree vision, or beat-'em-ups against bots with perfect reflexes. Playing against these AIs doesn't actually teach the player to play the game well: it teaches them to "game" the bots, to try to find a flaw in the algorithm which allows easy kills.
This isn't to say that all AI should be poor, but rather that good AI is multi-faceted. It makes mistakes, but tries to avoid falling into the same predictable patterns. Giving the AI the ability to adapt—and, therefore, to provide variance in gameplay—will likely provide much more entertainment for the player than an AI that plays the same perfect game every time.
Making Them Human
After making Infected, we briefly toyed around with a sequel, Infected 2. It was the same premise, but on a larger scale, similar to the classic board game Risk. Because the game was larger, it was important to have drastically improved AI: something which could keep the player entertained over the course of a much longer game, and where multiple AIs could compete against each other.
We plugged in the old AI, and there were several issues with it. The AI was predictable, but that was something that could be improved with a little work. The larger issue was regarding the general attitude of the AI, and how it played.
There were three main problems:
- Firstly, playing against the AIs wasn't particularly fun.
- Secondly, beating the AI wasn't particularly rewarding.
- Thirdly, the AI would "cheat" by deliberately throwing the game.
The game was fun against other players, but not against the AI. Although there was still the same level of challenge, why was there such a massive disparity between player against the AI and playing against another person, if the gameplay was the same?
We then remembered that humans have one thing computers don't: emotions.
Part of playing a game is getting emotional; we love a thrilling victory, we hate a crushing defeat. We love to rub our victories in our friends' faces, and we become petty and vengeful when they betray us. Games are intrinsically linked to our emotions, and it's something that an AI can never experience... but it is something that we developers can emulate.
The Vital Spark
Why wasn't playing the AI fun?
Because the artificial intelligence was so obviously artificial. All the bots played the same way; they all had the same approaches to the game; they were all equally predictable in their movement patterns.
We needed something to make them stand out from each other. So we introduced personalities: reckless, defensive, explorer. The AI would give an artificial weight to certain strategies, and then move in those attack patterns.
Even in a basic form, this was a massive improvement: no longer was every game the same, but it added unpredictability to enemy moves and made it harder to formulate a single "game winning" strategy. If the AIs were randomised when the game started, then the player wouldn't be able to predict how the game would unfold.
Personalities on AI is not a new idea. Civilisation gives personalities to its leaders (Gandhi famously loves to nuke people), Age of Mythology AIs determine how likely they are to use strategies such as turtling or rushing, and Unreal Tournament bots have favourite weapons.
Why Wasn't Beating the AI Rewarding?
The main issue was that the AI played what it viewed as a "best move" every round. It simply assessed the game board, with no regard to diplomacy. If it needed to attack you it would. If you attacked it back, it simply didn't care. It viewed the game as a mathematical puzzle to be solved, rather than a game against complex human players. Playing against our artificial intelligence resulted in an artificial game.
Our solution came from an Amiga game from 1989 called Nuclear War. In Nuclear War, you fought against various world leaders and tried to nuke their country from existence. Whoever managed to survive would win. Although simplistic, it was surprisingly fun watching your enemies get caught in personal vendettas and throw warheads at each other.
This "friends and enemies" system used a very simple metric: happiness. Each world leader had a variety of facial expressions which made it easy to judge their attitude towards you. Unsurprisingly, nuking people tended to make them unhappy very quickly. In addition, your enemies wouldn't just get upset with you, but with each other. This meant that a single wayward missile could escalate into a sustained fight to the death.
Once again, this system has been used in many games, including Civilisation. An enemy nation with a high level of "happiness" is more likely to ally with you and send you gifts, whereas an enemy that hates you might invest a lot of resources into crushing your cities.
By adding a rudimentary happiness meter into our game, we allowed friendships and rivalries to brew. A game is more exciting when you develop a nemesis, or when teams and alliances form. Suddenly, the game became personal, and actions felt like they had weight. We also found that a visible happiness meter was important, otherwise the player wouldn't understand what was happening. A simple happy face icon immediately conveys the idea that enemies can get "upset", and watching the face change upon taking certain actions made it clear what consequences these actions were having.
The prisoner's dilemma is a well known game theory concept, and many programmers have spent time trying to build the ultimate bot to compete in the iterated prisoner's dilemma. Curiously, the winning strategy for many years was also the simplest: known as tit-for-tat, it simply copied the opponent's move.
How Would the AI Cheat?
The problem in Infected 2 was often that the AI was simply looking at the short term perspective. The AI aims to maximise its score every turn: if this meant making an overall poor move, then it would do it anyway.
The easiest way to imagine it is this scenario:
- Alice, Bob and Carol are playing war.
- Alice has 100 soldiers. Bob has 75 soldiers. Carol has 150.
- Neither Alice nor Bob can beat Carol.
- However, Alice and Bob combined can beat Carol.
The problem was that the AI would do whatever would score it the most points. In the above scenario, that meant Alice would attack Bob: in the short term, an excellent move, but in the long term, disastrous. Not only is Alice making a move which dooms herself, she is condemning Bob to certain defeat as well. Bob is understandably upset.
This sort of situation is difficult to avoid, as seeing the bigger picture requires a complex reasoning beyond most simple AI. We tried to ensure "fairness" by making players less likely to be attacked if they were losing, or if they had been attacked recently. While not every game can apply a simple solution, its important to realise that players hate being attacked for no apparent reason.
While Infected 2 never made it past the alpha stage, it taught us some valuable lessons. Most importantly, that it's not enough to just play against enemy bots: you want to be involved in the game, develop rivalries, and crush all those who stand against you.
AI Done Right
Many years ago, a game was published that seemed leaps and bounds ahead of the competition. Although a multiplayer game by design, the bots were good enough that it could be enjoyed single player. And, although by today's standards it is poor, this game—Unreal Tournament—had AI which blew other games out the water.
We can look at an in-game screenshot to see some of what went into making a bot:
Skill was the main factor in determing bot behaviour: novice bots were slow, had to stand still to shoot, and were slow to turn around, while the higher level bots were faster, could dodge, had a bigger field of view and could use advanced weapon combo techniques.
Bots could then be futher modified with accuracy, alertness, camping, and strafing levels. "Personality" would be determined by combat style and favourite weapon, determining whether the bot liked to get up close and personal, or try and snipe from a distance. Bots would also taunt the player and seek revenge. Each of the 32 bots could be individually customised, meaning you could have 32 distinct personalities. Certain bots, such as Loque, became notorious amongst players.
All of these factors helped create bots which felt more human. While the game would still be good without this sophistication, it adds an extra, almost imperceptible layer of fun.
Conclusion
At the end of the day, you need to ask: what do you want from your AI? Are the enemies going to mindlessly rush the player and walk head-first into pits, or are you trying to develop something more, something that provides a more personal level of interaction with the player? Whatever you decide, remember three key points:
- If bots make mistakes, then they feel more human, and less like a "pack of cards".
- If they deliberately make plays based on individual personality (or emotion, like revenge), then unpredictable and exciting gameplay is more likely to emerge.
- If bots try to play cleverly, rather than short-sightedly, they are more likely to make the game fair to our eyes.
References
- Preview image: Ghost by Andres Flores from the Noun Project
- Preview image: Pacman by Andres Flores from the Noun Project
- Alice in Wonderland illustration: By John Tenniel, in the public domain
