QwenLM / BlenderWorld

BlenderWorld 
 Benchmarking Visual Agents for 3D Creation from Real-World References 
 Website ·
 Quickstart ·
 Evaluation ·
 Documentation 
 📑 Paper  | 
 🤗 Hugging Face  | 
 🤖 ModelScope 
*From real-world references to editable 3D scenes. The benchmark contains 223 tasks across 9 categories and 22 subcategories; references appear on the left/top and GPT-6-Astra outputs on the right/bottom.*
BlenderWorld evaluates agents on realistic 3D creation requests in Blender. Agents receive a task brief, reference media when applicable, and any declared specification files, then deliver an editable Blender project and the requested exports. Run your model with any generation harness, collect one submission ZIP per task, and evaluate it with `bw evaluate`. Harbor integration is available as an optional generation and verification adapter.
## Dataset
BlenderWorld contains **223 tasks and 2,651 binary criteria**, spanning products, architecture, environments, characters, animation, and scientific visualization. All tasks use neutral IDs from `bw_0001` to `bw_0223`.
| Input | Tasks |
| --- | ---: |
| Text | 78 |
| Image + text | 130 |
| Video + text | 15 |
| Total | 223 |
**59 tasks** also include specification attachments with requirements on dimensions, materials, or camera settings. These files can accompany visual references.
*Dataset statistics. (a) shows the 11 largest application scenarios, covering 213 of 223 tasks; (b) reference and specification types may co-occur; (c) the inner ring shows category and the outer ring shows importance.*
 How the benchmark is built 
*From production briefs to verifiable requirements. Diverse tasks and references undergo structural and semantic validation, followed by independent rubric review.*
## Quickstart
### 1. Install and download
Use Python 3.10+ on Linux. Download the dataset and check every task and resource:
 [code] 
To use an existing local dataset, pass its directory to `--dataset`. To pin a published revision, pass its commit SHA to `bw download --revision`.
### 2. Prepare, generate, and package
Prepare the inputs for one generation agent:
 [code] 
The prepared workspace contains `query.txt`, `input.json`, reference media, and declared attachments. Give the agent access to this workspace and Blender through its normal coding or MCP tools. Save the primary project as `model-output/final.blend`, alongside its requested exports and texture dependencies. Keep inputs and evaluation rubrics outside this output directory. Package its contents as one ZIP for the task:
 [code] 
The ZIP contains `final.blend` at its root, with optional `textures/` and `outputs/` folders. Ordinary ZIP tools are supported too. See the complete public submission format.
Generate projects with **Blender 4.2.10**. When using the bundled coding-agent environment, the other pins are **Node.js 22.17.0, Codex CLI 0.147.0, and Claude Code 2.1.220**.
### 3. Evaluate
Evaluation needs Blender 4.2.10, Node.js 22.17.0, ffmpeg/ffprobe, and the confi