A team of computer-controlled bots took on a team of top human players in a Dota 2 show match this weekend, and the computer didn't just beat the humans—it crushed them.
We were first introduced to OpenAI's Dota 2 bot at last year's International, the multimillion dollar Valve-hosted tournament that's the climax of the competitive season. Backed by Elon Musk, OpenAI's ambition is to ensure that when artificial intelligence is created it will be good for humans: think Lt. Cmdr. Data rather than Skynet or The Matrix. The OpenAI team is looking at Dota 2 because the game is enormously more complex than games like chess or Go. Unlike those games, Dota 2 is played with imperfect knowledge (you can't see the whole map at once, so enemy movements can be hidden), it's made up of thousands of moves over tens of minutes, and its goals are relatively long-term, making it hard to assess in the moment what action is best or which side has the advantage.
The bots learned to play Dota 2 through playing hundreds of years of matches against itself, previous versions of itself, and preprogrammed scripted bots. For last year's bot, the training was done on Microsoft's Azure platform with some 60,000 processor cores; this time around, OpenAI is using 128,000 cores on Google's Cloud Platform. The bots learn the game from scratch: initial versions will just wander aimlessly and at random as the game plays itself out. As thousands upon thousands of games are played, it figures out which actions will improve its chance of winning.
Last year's bot played a very limited form of the game: a single hero (Shadow Fiend) in a first-to-two kills one-on-one match. While a greatly simplified form of the game, it's nonetheless one that is regularly used in show matches to showcase individual player skill. Back then, the OpenAI bot soundly beat even top-tier human players. Over the course of many games, humans did manage to figure out strategies that the bot couldn't handle, showing one of the things that remains (for now) a uniquely human trait: the ability to improvise and undermine the spirit of the game, even while following the rules. Human players can operate on this meta-level, responding not just to the state of the game but also demonstrating that they understand the objectives and constraints of the game and then subverting them to their own end. The bot couldn't, leaving it exploitable.
Over the last year, the OpenAI team has been building a better bot: OpenAI Five. Dota 2 is a five-on-five team game, with 115 different playable heroes. OpenAI isn't yet able to handle that full game, but it's vastly more capable than it was: the bots can play 18 different heroes in five-on-five matches. Almost all of the game's items and mechanics are now supported, and the bot even handles drafting a team: picking heroes in response to the choices the human team has made to maximize its chances of success.
The human team was made of a number of former pros and casters, with a skill level estimated at putting it around the 99.95th percentile of all Dota 2 players. That's not quite at the level of a trained, coordinated professional team, but it's not far off. They were playing OpenAI Five in a best-of-three match.
The result? Well, it wasn't great for the humans. The bot won the first two games with ease, with a style of play that was heavily focused on objectives—pushing the enemy team's towers, controlling the map—and sustaining its heroes' health and mana during long engagements. This robbed the humans of the time they needed to accumulate the items and levels they needed to repel the push. The result was two quick victories, with both games ending in under half an hour, compared to the more typical 30-40 minutes for real matches.
The bots also did things that human teams don't. In typical Dota 2 teams, there's a prioritization of who should get the most gold; carry heroes (the main damage-dealers as the game goes on) are prioritized ahead of support heroes (ones with stuns and slows, used to control the pacing of fights and set up kills for their carries). Generally, OpenAI Five was much more generous to its support heroes than humans are. That's not to say that there's no prioritization—its carries were richer than its supports—but it's much less skewed than human teams prefer.
The bots are also much more willing to cast damage-dealing spells on any human players they happen to see. A particularly striking example was Sniper's Assassinate spell. This is a very long-range, high damage, single target spell: Sniper takes a moment to aim his gun, then fires a high-power shot. Human players typically use this to land killing blows, especially on enemies that are trying to flee a bad engagement. OpenAI Five's Sniper, by contrast, would cast Assassinate early on in fights, using it to bring support heroes' health very low and pick them off early on in team fights.
On the other hand, the bots show some surprising weaknesses: their last hitting (landing the killing blow on computer-controlled "creeps" yielding gold, the main income source within the game) was mediocre. In principle, the bots should last hit nearly perfectly: they can instantly determine the exact health of all the creeps and should know which creep to attack, and when, to maximize their income. Indeed, one of the hallmarks of programmed bots (rather than these machine learning-driven bots) is that they last hit nearly perfectly. Not so OpenAI Five, or at least, not yet.
Most important of all, the bots were expert trash talkers. The OpenAI Five bot calculates its win probability throughout the game, and, from time to time, it would all-chat this probability. In doing this, it showed that you don't need to be racist or sexist to effectively demoralize an enemy. The mere statement that "We estimate the probability of winning to be above 95 percent" is enough to crush the spirit.
With the bots victorious 2-0, things were shaken up for the third game. Instead of allowing OpenAI to draft its team, Twitch chat and the live audience were recruited to pick a line-up that was terrible: heroes who lacked many good slows and stuns to control games, heroes who all needed lots of items and levels to be effective, and heroes who were weak during the early stages of the game. The result? "We estimate the probability of winning to be 2.9 percent," the bots proclaimed. Even in spite of this unfavorable line-up, the bots put in a remarkably solid showing—at least to start out—with the humans taking more than half an hour to prevail.
This game also highlighted further weaknesses of OpenAI Five. At various points in the game, the computer-controlled Slark (a carry hero that's notoriously weak in the early game but late game can duck in and out of fights, with powerful regeneration powers and damage that increases the more he attacks enemies) behaved in decidedly unhuman ways to his detriment. Weak and outnumbered, what the Slark should have done was to hide in the trees, out of sight of the humans, and possibly teleport away without drawing their attention or wait for the rest of their team to come and help. Even low-level human players would have done this correctly, ensuring that they didn't die and could continue to accumulate gold to get their essential items. But the bot? It didn't know how to stay still. It would be safely hidden in the trees, but then, consistently, it would break cover and be killed off.
The three matches (and a fourth match against a low-level human team) can be watched here. The next outing for OpenAI Five will be the International in Vancouver later this month, when the computer will be taking on a real professional team for its ultimate test.
reader comments
131