NVIDIA adds four open-weight Cosmos 3 models for driving and robot simulation

NVIDIA has added four open-weight checkpoints to its Cosmos 3 model family, designed to generate interactive world simulations and multi-sensor, multi-view driving scenes, the company said in a post on Hugging Face.

The checkpoints are Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer and Cosmos3-Nano-Sim-Bimanual. NVIDIA said all four are post-trained examples of Cosmos 3 Nano with 16 billion trainable parameters, and that they use the OpenMDW1.1 license, which permits commercial and non-commercial use under its terms.

Driving models

Transfer-Auto is a bidirectional model for offline synthetic driving data generation, according to NVIDIA. It produces synchronized video across up to 11 camera views, including fisheye cameras, and supports joint RGB and LiDAR generation. Developers can vary weather, lighting and time of day, the company said.

Sim-Auto is the streaming version of Transfer-Auto, trained so each step depends only on earlier ones. NVIDIA said it produces camera video and LiDAR in successive short segments with only a handful of denoising passes, drawing on previously generated frames and fresh driving inputs to sustain long interactive runs. Closed-loop use requires a simulation runtime and a World Scenario Map renderer to update the controls, the company said.

Robotics models

Sim-Transfer takes spatial controls such as depth and segmentation from a physics simulator, plus a text prompt, and renders color observations a robot policy can consume. NVIDIA said physics and ground-truth state stay with the simulator.

Sim-Bimanual simulates how an environment changes in response to a two-armed robot's actions. NVIDIA said it supports YAM and AgiBot embodiments, controlled by a policy or human-operated joysticks, and generates the next observation directly from the scene and robot actions.

How the models work

Cosmos 3 uses a Mixture-of-Transformers backbone that links separate text-processing and visual-generation networks through attention, NVIDIA said. The models work on compressed video representations called latents, and the company said smaller chunks let new controls affect generation more often.

The post describes four training stages: bidirectional adaptation, teacher forcing, consistency distillation and self-forcing. NVIDIA said the final stage matters because a small prediction error about an object feeds into later frames as context, allowing mistakes to build up.

NVIDIA said the post-training recipes start from Cosmos3-Nano or compatible base weights. For Sim-Bimanual, it listed requirements of Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit, and said the published setup was tested on one GB200, which it said is not a minimum-memory specification.

The company acknowledged that hard problems remain. It said future work involves checking how well the models support policy learning and evaluation, mapping their limits, and improving weak areas.