We may not have flying cars, but we could well soon have robots that match up to predictions for the 21st century. Nvidia has announced GR00T, a cleverly named project to build robots using foundational models.

There aren't many clear details of what GR00T is all about or exactly who is involved, but you have to take Nvidia seriously as it has the hardware that makes much of modern AI possible. The whole thing seems to be very much linked to the new Jetson Thor SoC which includes a GPU based on the existing Blackwell architecture and a transformer engine. Transformers are the basis of foundational models.

GR00T stands for Generalist Robot 00 Technology and Nvidia is keen that we all know that it isn't pronounced GROOT because that would upset Disney. Instead we are to spell it out character by character.  Foundational models mostly get their knowledge of the world from human language, but couple this with basic training on robot movements in the real world and we have a system that can obey human commands. The real problem is getting enough training data and to this end Nvidia has released updates to its Isaac Sim application which allows robots to learn in a simulation. At the moment it seems that simulation is almost as important as the Jetson Thor platform.

Getting all this done is more work than simply announcing the name of the project. Nvidia is relying on some well-known partners to implement GR00T including  Apptronik, Agility Robotics, Boston Dynamics, Figure AI, Fourier Intelligence, and Sanctuary AI. We have already seen some moves towards AI control of Boston Dynamics robots and this could mean a whole new start for robot software. Gone are the physics-based motion controllers to be replaced by end-to-end foundational models. While we may well understand the physics- based controllers, we certainly will not completely understand their replacement AI. It looks like a lot of deterministic engineering code is about to bite the dust. This goes for self-driving car software as well since GR00T has a place for this sort of "robot" as well.

The key to the development of intelligent robots might just be the need to provide a body for the foundational models that we already have. Once a foundational model is constrained by the laws of the universe the problems of not being grounded in reality go away, along with the hallucinations that are so problematic at the moment. A general AI needs to be an embodied AI.

Here is a video about exactly what GR00T is. It may not be very informative, but it is impressive:

It looks as if things might turn out much as Isaac Asimov predicted in his I Robot stories. His positronic brain was a creation that made robots possible, but we had little idea how it worked in detail and, instead of engineers, a new breed of robot psychologist was needed to fix misbehaving robots. Sounds a lot like prompt engineering on steriods to me. 

And if you think this is all so unlikely, check out some of the research that has already happened about the meeting between foundational models and robots.

More Information

NVIDIA Project GR00T

Related Articles

RT-2 A Breakthrough For Robot Control

Google's Large Language Model Takes Control

ROScribe - LLMs At The Service Of Robotics

Spot + ChatGTP - It's Amazing

Should We Beware The Unitree Go2

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info