RoboDojo-Benchmark / RoboDojo

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies 
 Webpage | Document | Paper | Community | Leaderboard 
https://private-user-images.githubusercontent.com/88101805/619409345-cc074c5d-4567-4418-8a29-1385aaba9d5b.mp4
## News
- **September 16–17, 2026**
 - Fixed a single-frame discrepancy in `get_obs` and updated both the code and Hugging Face data. Rerunning evaluations across multiple models showed that the experimental results remain essentially unchanged.
 - Together with RoboTwin, fixed the channel ordering of the RGB data byte stream to keep both projects consistent. Please use this update with XPolicyLab commit `bb9a0b5` or later.
 - Updated the simulation assets for the `swap_T` task to make the objects less prone to slipping.
- **July 6, 2026** — Released the RoboDojo paper and open-sourced the code.
## ✨ Highlights
 Overview of RoboDojo. RoboDojo unifies efficient simulation evaluation and reproducible real-world testing for generalist robot manipulation, covering 42 simulation tasks, 18 real-world tasks, heterogeneous parallel simulation, RoboDojo-RealEval, XPolicyLab, and a continuously updated leaderboard. 
> RoboDojo is **eval-only** in this release: it provides the simulator client, benchmark tasks, asset/config validation, and result artifacts. Policy integration and policy servers are owned by XPolicyLab.
- 🌐 **Unified sim-and-real benchmark** — 42 simulation tasks and 18 real-world tasks across 3 robot embodiments for generalist robot manipulation.
- 🧭 **Five capability dimensions** — Generalization, Memory, Precision, Long-Horizon, and Open, designed to probe different skills rather than simple object or layout reskins.
- 🧗 **Challenging by design** — intentionally hard, diverse, long-horizon tasks that expose failures hidden by simpler benchmarks.
- ⚡ **Heterogeneous parallel simulation** — runs different tasks, scenes, and processes concurrently on Isaac Sim for fast, scalable feedback.
- 🧱 **Physically grounded assets** — rigid, articulated, and deformable objects in a single configuration-driven scene.
- 🤖 **Integrate once, evaluate everywhere** — XPolicyLab unifies 40+ policies behind one interface for both simulation and real-world runs.
- 📊 **Reproducible & leaderboard-ready** — seed-controlled layouts and one-command `summarize` aggregation into a leaderboard table.
## 📚 Documentation
The RoboDojo documentation is the canonical reference. Key sections:
| Section | Description |
| :-- | :-- |
| Usage Overview | End-to-end walkthrough of the evaluation workflow. |
| Installation & Downloading (Assets and Data) | Environment setup and downloading robot/object/layout assets/training data. |
| Quick Evaluation | Quickly dispatch XPolicyLab to run a policy for testing. |
| XPolicyLab | Integrates a large collection of policies and defines how to integrate new ones. |
| Simulation Tasks Details | The 42 Isaac Sim tasks across five capability dimensions. |
| Real Rob