Formula 1 Racing is an aggressive sport and competition can even become vicious. Through reinforcement learning Sony's racing AI, GT Sophy, has evolved a driving style that combines speed with "etiquette" and can outrace top human competitors. This breakthrough has now been recognized with an award.

The award is the 2022 ASM SIGAI Industry Award for Excellence in Artificial Intelligence which carries a cash prize of $5,000 and a plaque presented at International Joint Conference on Artificial Intelligence held last month in Vienna, Austria.

This annual award from the ACM Special Interest Group on Artificial Intelligence goes to individuals or teams who have transferred original academic research into AI applications in recent years in ways that demonstrate the power of AI techniques. The projects must illustrate this through a combination of features, including the originality of the research novelty and technical excellence approach; the importance of AI techniques to the approach; and the actual or predicted societal impact of the application.

In view of the final stipulation you might be surprised that the award has been made to team that has developed an AI agent that excels in Gran Turismo, the well known PlayStation 4 racing simulation game. 

Gran Turismo Sophy is the first AI agent to emerge from Sony AI. which was founded in 2020 with the mission to:

"unleash human imagination and creativity with AI".

It initially came to public attention when it was featured on the cover of Nature in February which carried the article Outracing champion Gran Turismo drivers with deep reinforcement learning authored by Sony AI's 27-person team, led by Peter Wurman, Director of Sony AI America. 

The paper describes how Sony AI trained agents for Gran Turismo that could challenge, and beat, the world’s best e-sports drivers.To quote from the abstract:

We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

There were two stages in achieving this success. By June 2021 GT Sophy was able to outrace top GT drivers when it was a head to head race on a empty track. In this video Emily Jones gives a commentary of her experience of being beaten by GT Sophy and what she learned as a result: 

The tables were turned once GT Sophy lined up against multiple human drivers in July 2021. In this scenario the program was at times too aggressive, racking up penalties for reckless driving, and at other times too timid, giving way when it didn’t need to. Retraining was required and by October 2021 GT Sophy won the rematch with ease.

The difference was due to adding a new dimension of driving skills. In addition to Race Car Control, the essential physics for car dynamics, racing lines and precision maneuvers and Racing Tactics, the split-second decision-making for passing and overtaking, training was provided in the Racing Etiquette deemed essential for fair play. To achieve this not only did the team have to find ways to encode the written and unwritten rules of racing into a complex reward function they also had to choose training examples for reinforcement learning that provided the correct balance of aggression and timidity.

Acquiring this skill is also what makes GT Sophy relevant beyond Gran Turismo. According to Wurman:

Etiquette between drivers on a track is a specific example of the kind of dynamic, context-aware behavior that robots will be expected to have when they interact with people, 

More Information

Sony AI

Gran Turismo Sophy

Outracing champion Gran Turismo drivers with deep reinforcement learning (Nature)

Related Articles

Why AlphaGo Changes Everything

Can DeepMind's Alpha Code Outperform Human Coders?

Robot cars - provably uncrashable?

The Triumph Of Deep Learning

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info