Google reveals Gemini Robotics 2.0, promising improved dexterity and safety - Ars Technica
Skip to content
AI
Biz & IT
Cars
Culture
Gaming
Health
Policy
Science
Security
Space
Tech
Forum
Subscribe
Story text
Size
Small<br>Standard<br>Large
Width
Standard<br>Wide
Links
Standard<br>Orange
* Subscribers only
Learn more
Pin to story
Theme
Search
Sign In
Sign in dialog...
Text<br>settings
Story text
Size
Small<br>Standard<br>Large
Width
Standard<br>Wide
Links
Standard<br>Orange
* Subscribers only
Learn more
Minimize to nav
Robots powered by Google’s Gemini AI models are now more capable. With the debut of Gemini Robotics 2, these physical bots can now accomplish more complex tasks, continuously analyze changing environments, and collaborate with other robots. This is thanks to a trio of new sub-models, one of which is publicly available for developers starting today.
Videos of robots running, dancing, and backflipping have been a staple of the Internet for years, but these machines were programmed to perform these very narrow tasks. The goal of Gemini robotics is to create a generalist robot, one that can do anything a human could do. Google DeepMind scientists sometimes call this “physical AGI.” Essentially, you tell a robot what to do, and it does it. With the 2.0 release, Google says its robotics AI can control an entire humanoid robot with improved dexterity, even for machines with complex humanoid hands.
Gemini Robotics 2 brings whole-body intelligence to robots.
This starts with Gemini Robotics ER 2, an upgraded “embodied reasoning” model that DeepMind claims is a significant leap over the previous 1.6 release. It’s integrated with the Gemini Live API, giving developers the opportunity to experience that supposed leap forward.
Gemini Robotics ER 2 is what’s known as a vision language model (VLM). It’s designed to understand instructions and the world around it. The big upgrade here is that ER 2 can process live video feeds from the robot’s cameras, allowing the system to track progress as the robot lumbers from one step to the next. Google notes that Gemini Robotics ER 2 can classify video frame completeness with almost 60 percent accuracy. That’s still far from perfect, but it’s much better than the 1.6 release or what you can get with the visual understanding of competing AI models.
Gemini Robotics ER 2 is better at understanding the world than other models, but not by much.
Credit:<br>Google
Gemini Robotics ER 2 is better at understanding the world than other models, but not by much.
Credit:
Finding specific moments in video feeds is also key to completing a task correctly. When you ask a robot to pour a cup of coffee, you definitely want it to know when to stop pouring. ER 2 apparently does this much better, identifying key moments with almost 90 percent accuracy. As the robots execute multi-step tasks, the embodied reasoning model allows them to understand failures in real time. The system can then attempt that single step again rather than going back to the start. For example, the robot can just readjust its hand position and motion if a ball it’s trying to pick up rolls away or someone moves a container.
The new embodied reasoning release is also what gives Google DeepMind’s upgraded robot AI the ability to collaborate. The video demos show Apptronik’s Apollo 2 and the simpler Franka F3 Duo working together on a task without getting in each other’s way. While the test robots still can’t match the speed or grace of a human, the video demos include plenty of real-time footage of the robots in action, and they do seem much less hesitant than they were in past tests.
Multi-robot collaboration with Gemini Robotics 2.
Understanding is only the first step—getting the robot to move around in the physical world is the purview of another model. After mapping out the task, the vision language model hands things over to an upgraded vision-language-action known simply as Gemini Robotics 2. This AI model generates robot actions from those instructions in the same way other generative systems create text or images. There’s also a low-latency offline version of this called Gemini Robotics On-Device 2. These models are currently limited to a small group of testers.
Google is also testing Gemini Robotics 2 on Boston Dynamics hardware.
Google is also testing Gemini Robotics 2 on Boston Dynamics hardware.
Google says the new action models are much more accurate and efficient. Even the smaller on-device version can adapt to new robot designs with just a few hours of movement data, or around 200 examples.
Heading off the robot apocalypse
The issues with AI hallucinations are well known at this point, but the potential harm from mistakes when the AI has a physical embodiment sharing space with humans could be much greater. With each release of Gemini Robotics, Google DeepMind has stressed that it takes this risk seriously. According to DeepMind, each layer in Gemini Robotics includes...