NVIDIA Puts a 4B World Model on the Robot - by Jay Chia

SubscribeSign in

NVIDIA Puts a 4B World Model on the Robot<br>World models are usually too heavy for the control loop and run on a server. Cosmos 3 Edge is small enough to run on-device.<br>Jay Chia<br>Jul 23, 2026

Share

Cosmos 3 Edge, running on a Jetson

Cosmos 3 Edge is a 4B world foundation model built as a Mixture-of-Transformers, pairing an autoregressive reasoner with a diffusion generator that produces the action chunk. NVIDIA reports it running at roughly 15 Hz on a Jetson Thor at 640×360 and 32 actions per inference, generalizing across humanoids, grippers, and driving. The release includes open weights, a Cosmos-3-Edge-Policy post-trained on DROID for pick-and-place, and the post-training scripts to adapt it to your own setup. The numbers are NVIDIA’s own, announced at SIGGRAPH, and the open question is whether a model small enough for the edge is good enough to drive a policy directly instead of standing in as a planner or offline evaluator.

Thanks for reading Topic Queue! Subscribe for free to receive new posts and support my work.

Subscribe

Research

Xiaomi-Robotics-1 scales a VLA on 100K hours of real trajectories

Xiaomi-Robotics-1 pre-trains a Mixture-of-Transformers VLA on over 100,000 hours of UMI trajectories, using a VLM to auto-caption scene-state transitions, and reports clean scaling curves that carry into real-robot evaluation. It claims a new best on RoboCasa365 (57.6%, up from 46.6%) on the authors’ own evaluations, though code and weights are still marked coming-soon.

RoboTTT makes context length a scaling axis for policies

RoboTTT, from NVIDIA GEAR and Jim Fan’s group, folds test-time training into a robot policy to stretch visuomotor context to 8,000 timesteps with no added inference latency, reporting an 87% gain on long-horizon multi-stage tasks. The argument worth engaging is that pre-training context length behaves like a clean scaling axis for robot foundation models.<br>Patch Policy drops the VLM backbone

Patch Policy, from Yann LeCun and Lerrel Pinto’s group at NYU, feeds a policy-dense pretrained ViT patch tokens through a block-causal mask, keeping the benefits of large-scale visual pretraining without a full VLM. It reports beating a fine-tuned OpenVLA-OFT by 18% while using about 0.7% of the parameters, the kind of efficiency result a lab can adopt directly.<br>Masked Visual Actions turns a video model into a world model

Masked Visual Actions, from Fei-Fei Li’s group at Stanford, expresses an action as a partially revealed trajectory of any entity in a video, so one fine-tuned video checkpoint serves as forward model, policy evaluator, and inverse model. It is trained on 15 hours of masked examples, and the imagined rollouts correlate with real execution well enough to use for evaluation.<br>WANDA generates an open-world dataset from a single demo

WANDA, from Guanya Shi’s group at CMU, takes one real RGBD demonstration, reconstructs the scene with Gaussian splatting, then rearranges the contact-rich segments through whole-body motion planning to synthesize photorealistic training data across new layouts and environments. Policies trained on it generalize spatially and across environments from that single demo, and the authors show zero-shot transfer to a mobile manipulator with a different morphology; they report it outperforming policies trained on 50 teleoperated demonstrations.<br>Open-AoE opens a 2,000-hour egocentric manipulation dataset

Open-AoE, from Microsoft Research, is a roughly 2,000-hour egocentric manipulation dataset collected by 500+ contributors on 400+ phones, with hand poses, camera trajectories, and temporally localized atomic actions. It ships the full capture-to-training pipeline and a cross-embodiment retargeting toolchain, which is the reusable part for anyone running a human-video data operation.<br>REAL puts an open-world manipulation agent on a real benchmark

REAL, from InternRobotics, is an agentic mobile-manipulation framework with sim-to-real-consistent APIs and a simulated user for intent disambiguation, released alongside REAL-Bench, a 241-task benchmark covering exploration, visual distraction, and articulated manipulation. It reports 56.9% on interactive tasks against closed VLMs and 78.3% over 60 real-world episodes, with code out.<br>Sunday Robotics previews ACT-2 and a reliability standard

ACT-2 is Sunday Robotics’ home-robot model, previewed with a 99.1% zero-shot success rate folding laundry across unseen homes over 785 attempts, and a claim that a single fine-tuning example can teach a behavior that generalizes. The more interesting contribution is a proposed evaluation standard they call a Solve, reliable performance across a declared scope at a stated adaptation cost, though this is a company blog with self-reported numbers and no paper or weights.<br>Quick hits

RoboInter1.5 — a 230k-episode suite of dense intermediate representations plus a VQA benchmark and a world model, aimed at...