mll-lab-nu / embodied-first-exam

Embodied First Exam 
 Code agents have passed many exams in the digital world. 
 What is their first exam to enter the physical world? 
 Our nine original suites: RoboPaint · RoboARC · RoboDynamicExam · Embodied Olympics - Fun · Embodied Olympics - Match · DexHandExam · RoboEngineer · Embodied Safety Exam · iSpatialGym. ▶ Watch the intro video 
## News
- 🔥 **2026-10-19**: Embodied First Exam is released!
## Framework
A coding agent writes code, runs the robot and sees the result, in a frozen scene from one of the suites, in each mode.
## Quick start
### Before you run
- **Host:** Linux x86_64 with git, uv (it fetches a Python 3.12-3.14 for Harbor if needed) and Docker Engine with Compose v2 and BuildKit (the `buildx` plugin). Without Docker, e.g. on a Slurm cluster: rootless podman with `-e podman` (docs/protocol/running.md).
- **GPU:** the NVIDIA driver and the NVIDIA Container Toolkit for every task that uses a GPU. An RTX-class GPU (with RT cores; not A100 or H100): AM-Bench, BEHAVIOR-1K, iSpatialGym, ManiGuard, RoboARC, RoboDojo, RoboLab, YAMLab. Any NVIDIA GPU: BiGym 2.0, DexToolBench, MolmoSpaces, Panda Dynamic Scenes, RoboPaint, RoboTwin 2.0, RoboWits. Some tasks: Embodied Olympics - Fun, Embodied Olympics - Match, HumanoidBench, IKEA MuJoCo, KinDER, MuJoCo Playground, MyoSuite, roboEngineer. No GPU: DexHand, DexJoCo, Embodied Safety Exam, LIBERO-MAX, MetaWorld+, RoboCasa, RoboCasa-GR1, RoboCasa365, VLABench.
- **Disk:** a benchmark's public images take 1-30 GB, pulled on its first run; a benchmark that builds its image needs the free disk its row below states.
- **API keys** for the agent's model in `.env` (step 1), or a subscription (step 3).
These benchmarks need a step by hand, or build their image on their first run because it may not be redistributed. That build accepts the licences of what it installs on your behalf: read them in the linked section. Organization members logged in to ghcr.io pull the prebuilt layers instead (on podman, after `podman login ghcr.io`, `harbor run -e podman` pulls the prebuilt image and uses it in place of the recipe; docs/protocol/running.md).
| Benchmark | By hand, once | First run |
|---|---|---|
| AM-Bench | - | Builds its image (accepts NVIDIA's Isaac Sim terms): about 25 min, 60 GB free disk. |
| BEHAVIOR-1K | Download the BEHAVIOR dataset (its downloader asks you to accept the BEHAVIOR Data Bundle licence; about 32 GB, 40 GB on disk) and export `RCB_BEHAVIOR_DATA`. | Builds its image (accepts NVIDIA's Isaac Sim terms): about 20 min, 30 GB free disk. |
| iSpatialGym | The BEHAVIOR-1K dataset, `RCB_BEHAVIOR_DATA` (the same directory). | Builds its image (accepts NVIDIA's Isaac Sim terms): about 20 min, 31 GB free disk. |
| LIBERO-MAX | A CPU with AVX-512 (Intel Ice Lake or newer Xeon, AMD Zen 4 or newer). | - |
| ManiGuard | Prepare the data root `RCB_MANIGUARD_DATA` (3.4 GB; its key asks you to accept the BEHAVIOR Data Bundle licence). NVIDIA driver 580.x: newer drivers crashed at start. | Builds i