An Interactive World Simulator Trains and Evaluates Policies for Robots

An Interactive World Simulator Trains and Evaluates Policies for Robots
3 min read

Researchers present an ‘Interactive World Simulator’, a simulator that predicts video images in response to a robot’s actions, without a physics engine. According to the authors, this tool could reduce two bottlenecks in robot learning: the collection of training data and the evaluation of policies, meaning the programs that control robots.

Today, the common method involves collecting hundreds of demonstrations performed by an expert on a real robot, training an imitation policy on this data, and then testing it repeatedly on the same robot. These steps are slow, costly, and difficult to reproduce, as hardware can fail, lighting can change, and objects can move. Classic physics-based simulators offer an alternative, but they require manually modeling geometries, contacts, friction, and deformations, and often remain too far from reality for policies trained in simulation to then work on a real robot.

A model that learns from actions

The authors chose a different approach. Their simulator is a video prediction model, conditioned by the robot’s actions: from the current image and a sequence of actions, it generates the following images directly in pixels, without a physics engine. It was trained on four manipulation tasks: pushing a T-shaped object (contact between rigid bodies), threading a rope through a clip (interaction between a deformable object and a rigid object), grasping a cup (fine gripper dynamics), and sweeping a pile of objects.

According to the authors, an operator can control the robot through this model for over 10 minutes at 15 FPS, or 15 frames per second, on a single RTX 4090 card, while obtaining video that remains stable and physically plausible. To achieve this, the system operates in two stages. A first autoencoder compresses RGB images into compact latent representations, then reconstructs them into images. A second model, for which the autoencoder is frozen, predicts the next latent state from past latents and actions, before decoding into an image. This prediction repeats autoregressively, with each produced image serving as context for the next. The authors indicate they use consistency models to keep the simulation interactive and stable over time.

Two uses: generating data and evaluating

In a first experiment, the authors collected demonstrations entirely within the simulator, then trained imitation policies with 0% real data and 100% generated data. These policies, deployed directly on the real robot, accomplish the tasks and remain robust against continuous human perturbations. According to the authors, this shows that the quality of the generated data is comparable to that of real data.

In a second experiment, four policies, DP, ACT, π0 and π0.5, were evaluated both in the simulator and on the real robot, with the same initial configurations. Each evaluation point corresponds to a policy checkpoint, scored in both environments. The authors observe a strong correlation between the two series of scores, across tasks and policies. Qualitatively, policies that succeed in simulation also succeed on the real robot, and those that fail in simulation also fail in reality.

The authors indicate they want to extend their framework to more varied environments and more complex manipulation tasks. They also cite studying how the performance of these models evolves with the amount of interaction data and computational resources, a question they deem useful for designing larger world models for robotics.

Yixuan Wang is a final-year PhD student in computer science at Columbia University and a research scientist at the NVIDIA Seattle Robotics Lab. The project is presented on a dedicated page, where the simulator can be tested in a browser via keyboard.

S’abonner
Notification pour
guest

0 Commentaires
Le plus ancien
Le plus récent Le plus populaire
0
Nous aimerions avoir votre avis, veuillez laisser un commentaire.x