THUSI-Lab / PrismWAM

# PrismWAM
**Decoupling State Representations in World Action Models**
🚀 Quick start · 🔥 Training · 🎬 Evaluation & deployment · 📦 Models
🌐 **Project page** · 📄 **arXiv** · 🤗 **Hugging Face**
## News
- **2026-10-23** · PrismWAM\* ranks 🥇 **#1** on the official **RoboTwin 2.0 leaderboard**
 (**83.67** average, **75.50** clean → randomized) and 🥈 **#2** on **RoboDojo** (w/o Agent track),
 ahead of models pretrained on thousands of hours of robot actions.
- **2026-10-19** · Released code and training recipes, with **PrismWAM** checkpoints for RoboTwin 2.0, RoboDojo and LIBERO on Hugging Face.
## Overview
Video latents entangle *how the world looks* (texture, colour, lighting) with *how it is
physically configured* (geometry, pose, contact), yet actions change only the latter.
PrismWAM decouples the two along a **Semantic Planning → Physical Rollout → Executable Action** chain:
- 🎯 **Semantic planning with sparse RGB.** Predict a single future RGB frame, the goal at the
 end of the action chunk. It teaches *what* to achieve and is only a training target, so inference generates no future RGB.
- 🧊 **Physical rollout in dense depth.** Predict every future depth frame. Depth keeps geometry
 and motion but drops texture and lighting, so it is easier to predict.
- 🦾 **Executable action from the rollout.** An action expert reads the depth rollout through
 one-way RGB → Depth → Action attention and is denoised jointly with it.
Results without any embodied action pretraining:
| Benchmark | Setting | PrismWAM | PrismWAM\* |
| --- | --- | ---: | ---: |
| RoboTwin 2.0 | Clean → Randomized | 70.57 | 83.67 |
| RoboDojo | Score / SR | 25.10 / 18.55 | 36.23 / 29.30 |
Success rates in %, except the RoboDojo score. Clean → Randomized trains on clean scenes and
tests on clean and randomized ones. Comparisons with other models are on the
project page.
The LIBERO model is also tested on LIBERO-Plus and LIBERO-Pro, perturbed versions of LIBERO,
without further training (success rates in %):
| Benchmark | Setting | PrismWAM |
| --- | --- | ---: |
| LIBERO | 4 suites | 99.35 |
| LIBERO-Plus | Trained on LIBERO | 79.44 |
| LIBERO-Pro | Trained on LIBERO | 54.48 |
## Quick start
Install (Linux, NVIDIA GPU, CUDA 12.8 toolkit; see installation):
 [code] 
Load a released policy. The model and the T5 text encoder that reads its instructions are
downloaded to `weights/` on first use:
 [code] 
See the observation and action format.
## Model zoo
| Model | Benchmark | Trained on | Recipe | Download |
| --- | --- | --- | --- | --- |
| PrismWAM | RoboTwin 2.0 | 50 tasks, clean scenes | `robotwin` | 🤗 robotwin |
| PrismWAM | RoboDojo | 35 tasks, ARX X5 | `robodojo` | 🤗 robodojo |
| PrismWAM | LIBERO | 4 suites, 40 tasks | `libero` | 🤗 libero |
| PrismWAM\* | RoboTwin 2.0 | 50 tasks, clean scenes | — | 🤗 PrismWAM-Star robotwin |
See the model guide for downloads and model layout.
Training initializers are available in `init/`.
## Training
Two stages per benchmark, one YAML recipe and one