AMAP-ML / Urbex

# Urbex
Official implementation of the paper "Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes".
Urbex formulates city-scale 3D grounding as active spatial search, enabling a vision-language agent to progressively gather evidence in an interactive multi-view environment without precomputed instance proposals. Reinforcement learning jointly optimizes localization and search decisions, teaching the agent where to inspect, how to refine its search and when to commit an answer.
## Installation
Requires Linux, Python 3.11 and the CUDA 12.8 toolkit (`nvcc` and a C++ compiler). Run commands from this repository root.
 [code] 
The installer includes Megatron Core 0.16.0, Transformer Engine 2.9.0 (CUDA 12 / PyTorch bindings), FlashAttention 2.8.1 and NVML. The training launcher locates PyTorch's cuDNN libraries automatically.
## Data and model
Try the Colab notebook for single-query inference with an interactive 3D view of the prediction. The model weights and data are on Hugging Face; the dataset card lists the terms of the upstream datasets.
 [code] 
Use `--scenes ...` for selected scenes, or `--candidates` for CityRefer Hit@k. Data files are checksum-verified; `--repo-id` or `HF_DATASET_ID` / `HF_MODEL_ID` selects another Hub repository.
Original point clouds are needed only for 3D metrics and asset building. For STPLS3D, add `--point-clouds` to download the PLY into the sibling `raw_data/` directory. SensatUrban point clouds (CityRefer scenes) are not redistributed: request them through the official SensatUrban release and place the PLY files under `raw_data/sensaturban/ply/{train,test}/`.
BEV images and oblique-view caches are provided. Build them only as needed:
 [code] 
CityRefer merges scenes across available annotation splits. Use `stpls3d` for STPLS3D, or add `--scenes ...` to build selected scenes.
## Training
 [code] 
Training uses Megatron and vLLM. The default GRPO configuration uses 8 GPUs; adjust device mappings and parallel sizes for your hardware.
## Evaluation
 [code] 
Set `CANDIDATE_DIR` to the candidate pool for Hit@k, and `URBEX_RAW_DATA` to the raw point-cloud directory for 3D metrics. Results are saved under `output/`. The rollout configuration controls inference settings.
## Citation
 [code] 
## Acknowledgements
Built on ROLL. See LICENSE and NOTICE.