A team comprising five artificial intelligence agents that had learned to play Dota 2 from scratch simply by exploring and experimenting, has convincingly beaten a team of five human player, four of whom are top ranked Dota 2 professionals.

Last year a single OpenAI Dota 2 bot made a surprise appearance at Valve's International Dota 2 tournament held in Seattle and had a decisive victory playing against Danylo “Dendi” Ishutin, who having been beaten in the first two rounds declined to play the third. It had taken just two weeks for that bot to go from knowing nothing about Dota 2, the battle arena video game from Valve in which the goal is to destroy the oppostion's large structure known as the Ancient while defending their own.

Having experienced success in the Dota 1v1 mini-game, OpenAI went on to train bots to work together in a team of 5, the usual format for playing Dota 2. The team of five neural networks, OpenAI Five plays 180 years worth of games against itself every day in order to take on the best human players at the full game.

According to Open AI:

It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores — a larger-scale version of the system we built to play the much-simpler solo variant of the game last year. Using a separate LSTM for each hero and no human data, it learns recognizable strategies. This indicates that reinforcement learning can yield long-term planning with large but achievable scale — without fundamental advances, contrary to our own expectations upon starting the project.

In preparation this year's Valve International 8 later this month has an OpenAI Five Benchmark match was held in San Francisco on August 5th. This was a best-of-three against a team of 99.95th percentile Dota players: Blitz, Cap, Fogged, Merlini, and MoonMeander — four of whom had played Dota professionally — in front of a live audience and 100,000 concurrent livestream viewers. There were still some restrictions to the game:

• Pool of 18 heroes - rather than 113
• No Divine Rapier, Bottle
• No summons/illusions
• 5 invulnerable couriers, no exploiting them by scouting or tanking
• No Scan

Prior to the Benchmark Game a team of volunteers for the audience played a public match against OpenAI Five. Five won within 14 minutes whereas an evenly matched game would take around 45 minutes. 

In the Benchmark match itself, a new OpenAI Five capability was revealed - drafting, something that is considered an extremely challenging part of Dota, since heroes interact with each other in complex ways. This was an implemented as an extension of adding a win probability output to the neural network to introspect what OpenAI Five is predicting.

The latest OpenAI blog explains:

When later considering drafting, we realized we could use this to evaluate the win probability of any draft: just look at the prediction on the first frame of a game with that lineup. In one week of implementation, we crafted a fake frame for each of the 11 million possible team matchups and wrote a tree search to find OpenAI Five’s optimal draft.

Reporting the results of the Benchmark match the blog states: 

After the game 1 draft, OpenAI Five predicted a 95% win probability, even though the matchup seemed about even to the human observers. It won the first game in 21 minutes and 37 seconds. After the game 2 draft, OpenAI Five predicted a 76.2% win probability, and won the second in 24 minutes and 53 seconds.

Given that OpenAI Five had already won, for game 3 the audience was invited to draft OpenAI Five's heros and chose an adversarial lineup that was described as "fairly Looney-Tunes".

Open AI Five predicted a 2.9% chance of winning, later improving this to 17% but lost after 35 minutes and 47 seconds.

This OpenAI video is intended to provide some insight into the model’s planning via an output which predicts where a hero will be in the future. In it the highlighted boxes show the predicted location of Sven in 6 seconds:

OpenAI Five continues to prepare for playing a team of professionals at The International. In terms of compute time the Auguest 5th model uses 190 petaflop/s-days, which is a lot of number crunching.

More Information

OpenAI Five Benchmark: Results

OpenAI Five

Related Articles

OpenAI Bot Triumphant Playing Dota 2

OpenAI Recruiting Fellows

AI Goes Open Source To The Tune Of $1 Billion 

Open AI And Microsoft Exciting Times For AI 

OpenAI Universe - New Way of Training AIs

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info