Orchestrate a Spot robot through multi-step tasks
Gemini Robotics ER 2 drives a Boston Dynamics Spot, watching video to confirm tasks like tying a trash bag are done before moving on.
What they did
Developers declare low-level controls, such as Boston Dynamics Spot's navigation and manipulator APIs, as tools. They stream video, audio or text into Gemini Robotics ER 2 through the Gemini Live API. The model sequences the steps, watches the video to track progress and confirm a task is finished, then moves on. The published Spot demo fetches objects, such as a popcorn snack, on a spoken command.
How it went
On progress classification, which sorts each video frame into five progress bands, it reached 57.4% accuracy. On moment-finding it reached 91.3% accuracy with a 0.96s mean absolute distance. The post gives no Spot success rate for tying a trash bag.
Worth knowing
The Spot demo code is on GitHub, and the model is available as a preview through the Gemini API and Google AI Studio.
Labels
The published Spot demo fetches objects, such as a popcorn snack, on a spoken command.
The model sequences the steps, watches the video to track progress and confirm a task is finished, then moves on.
the model is available as a preview through the Gemini API and Google AI Studio.
The “Material” label failed its quality check and is not used for selection.
Details
- Agents
- Gemini
- Source
- other
- Source date
- undated
- Accounts
- 1
- In registry since
- Oct 12, 2026