UniWAM / UniWAM

# UniWAM: Unified World-Action Model
## 📃 Overview
UniWAM is an 8B-parameter model that brings semantic understanding, visual prediction, and action generation into one MoT architecture. Its three experts exchange information through joint multimodal attention. During pretraining, physical-language supervision and complementary signals from robot, human egocentric, and VQA data build embodied knowledge. During post-training, future visual noise augmentation reduces reliance on precise future predictions, while encoded action history initializes action generation through flow matching.
## 🌟 Key Features
- **Unified world-action architecture:** A Mixture-of-Transformers (MoT) connects a physical reasoner, a world generator, and an action predictor through joint multimodal attention.
- **Physical-language grounding:** Robot actions are represented in natural language, adapting the vision-language component to embodied tasks while retaining its language capabilities.
- **Complementary multimodal supervision:** The pretraining recipe combines robot demonstrations, human egocentric data, and visual question answering (VQA) data to train the appropriate experts.
- **Efficient action generation:** Future visual noise augmentation and history-conditioned flow matching support action generation with fewer denoising steps.
- **Efficiency:** Post-training on RoboTwin 2.0 takes 10 hours on 8 NVIDIA H100 GPUs.
## 📚 Contents
- `models/`, `train/`, `utils/`, `bak/wan/`: physical language model and training runtime.
- `data/robotwin2/`: RoboTwin loader and conversion utilities (code only).
- `data/libero/`, `examples/libero_plus/`, `scripts/libero/`: LIBERO training and LIBERO/LIBERO-plus evaluation code.
- `configs/`: RoboTwin physical language, IDM, history-flow, and future-noise examples.
- `inference/robotwin/uniwam/`: self-contained RoboTwin policy deployment.
Bridge, DROID, Fractal, and real-world inference are intentionally out
of scope for this release.
## 🚀 Installation
Python 3.10 and a CUDA-capable PyTorch installation are recommended. Install
PyTorch for your CUDA version first, then install the remaining dependencies:
 [code] 
## 💾 Model weights
### Pretrained assets for training
Standard training uses the following pretrained assets:
| Pretrained asset | Link | Fine-tuning data or role |
| --- | --- | --- |
| ` UniWAM-base` | ModelScope / Hugging Face | pretained checkpoint on mixed robot,human and VQA datasets. |
| ` UniWAM-robotwin-clean` | ModelScope / Hugging Face |post-trained checkpoint on the clean subset of RoboTwin 2.0. |
| `Wan-AI/Wan2.2-TI2V-5B` | Hugging Face | Video backbone, VAE.|
| `Qwen/Qwen3-VL-2B-Instruct` | Hugging Face | Vision-language backbone;. |
Keep the downloaded directory structure as follows so that it matches the
default paths in the training configs:
 [code] 
## 🚀 Post-training
### RoboTwin
See the RoboTwin post-training guide
for data preparation, training, and inference.
### LIBERO
See the LIBERO guide for original LI