ZJU-OmniAI / Scene-Designer

Scene-Designer 
 Learning Design Intelligence for Agentic 3D Scene Generation 
 Xuanyu Liu*, Hongyan Feng*, Miao Pan, Sunlai Chen, Lizhou Liang, Bing Sun, Wenqi Zhang, Xuhong Zhang 
 Zhejiang University · Universal Identification Technology (Hangzhou) Co., Ltd. 
 OmniAI Group of ZJU ACES Lab * Equal contribution 
## 🎬 Introduction Video
Open video directly · Watch on the project page
## 📖 Overview
**Scene-Designer** learns **design intelligence**: the ability to both **design a scene** and **critique its quality**.
**Scene design** concerns constructing physically valid, functionally correct, and layout-coherent scenes, while efficiently correcting errors without unnecessary trial and error when problems arise. **Scene critique** focuses on assessing scene quality along the same dimensions and distinguishing better designs from poorer ones. These two capabilities are closely connected, since effective design requires reliable critique to guide construction, while meaningful critique requires an understanding of what constitutes good scene design.
## 🧭 Learning from Design and Critique Experience
Two complementary sources of supervision teach construction decisions and scene-quality assessment:
- **Scene-OTA-35K:** Observation–Thought–Action experience. Causal Trajectory Consolidation removes ineffective operations without downstream contributions while retaining effective decisions and necessary intermediate states.
- **Scene-OTC-15K:** Observation–Thought–Critic experience. Scene perturbations introduce layout deviations, functional degradation, and physical violations to construct scene preference comparisons.
## 🧠 From Scene-Builder to Scene-Designer
Joint supervised fine-tuning develops scene construction and critique capabilities, producing **Scene-Builder**. **Scene-GRPO** then uses a frozen Critic to evaluate candidate edits through two complementary local rewards:
- **Branch-ranking reward:** compares sibling scenes generated from the same parent to favor better design decisions.
- **Improvement reward:** compares each child scene with its parent to assess actual progress and discourage ineffective modifications.
Global length and object-completeness rewards support efficient, complete scene construction.
## 📦 Models and Data
| Resource | Description | Access |
| :--- | :--- | :---: |
| **Scene-Builder-Qwen3-VL-8B-Instruct** | Full SFT checkpoint, initialized from Qwen3-VL-8B-Instruct | 🤗 Model |
| **Scene-OTC-Bench** | 94 two-scene comparisons and 55 four-scene ranking questions | 🤗 Dataset |
The published weights are the **Scene-Builder SFT stage**, before Scene-GRPO. Scene-OTC-Bench is held out for evaluation; do not mix it into training.
## 🚀 Quick Start
### 1. Clone the repository
 [code] 
### 2. Prepare the environment
Use Linux and Python 3.11. Follow the installation guide to install the model dependencies and prepare the separately downloaded **SceneSmith** checkout, integration patch, geometry services, and assets.
SceneSmith is