amap-cvlab / ABot-Recon

# ABot-Recon
## Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction
English | 中文
> **In one sentence:** ABot-Recon reconstructs long video streams with a fixed 12-frame local context, composing current-frame geometry and adjacent relative poses into a global reconstruction without persistent learned long-range memory. 
## 📣 News
- **2026-10-14:** We have added visualization scripts for rendering inference results into point-cloud videos and bird's-eye views of camera trajectories and point clouds. See the `visualization` folder for usage and configuration.
- **2026-09-25:** Fine-tuning code is now on the `train` branch, with 18 data sources, mixing/EMA/validation/resume, and example configs. See the branch README to get started.
- **2026-09-16:** Community port: The AXERA-TECH team has ported ABot-Recon to the Axera AX650N NPU, supporting both AXCL-based PCIe inference and on-device inference on AX650N boards. The release includes precompiled AX650N models, Docker images, and a web-based mapping service. Model files are available on Hugging Face. Thanks to the AXERA-TECH team for this hardware adaptation!
- **2026-08-31:** Thanks to the Hugging Face team, an interactive ABot-Recon Demo is now available online. Try it out!
## Why local context?
Long-horizon streaming reconstruction is often approached by adding increasingly elaborate mechanisms for retaining and fusing long-range state. ABot-Recon takes a deliberately local route. At each time step, it solves the same bounded prediction problem:
- cache KV features from the preceding 11 frames;
- predict a point map $P_i$ in the current camera coordinate system;
- estimate the adjacent relative pose $T_{i-1\leftarrow i}$; and
- recover the global trajectory and point cloud through sequential pose composition.
This design keeps model-state memory and per-frame computation independent of the elapsed sequence length. A lightweight motion-visual rotation refiner and composition-aware pose loss are used to limit drift when local poses are composed over long horizons.
## Results at a glance
| Evaluation | Result | Setting |
|---|---:|---|
| Oxford Spires camera pose | ATE **4.35 m**, RPE-R **0.12°** | Streaming model only; no loop closure |
| Oxford Spires dense reconstruction | CD **1.37 m**, F1 **91.81%** | F1 threshold $\tau=4$ m |
| KITTI-02 streaming efficiency | **24.45 FPS**, **6.71 GiB** | 504×280, NVIDIA H100, input storage excluded |
The full paper reports camera-pose results on KITTI, Oxford Spires, and VBR, together with dense reconstruction on 7Scenes, TUM-Dynamic, and Oxford Spires.
## Installation
The released configuration targets Linux, Python 3.10 or later, PyTorch 2.5.1, and CUDA 12.1. The release environment was validated on NVIDIA A100, while the paper's runtime benchmark uses an NVIDIA H100.
 [code] 
### Recommended acceleration
ABot-Recon uses paged KV-cache operators from FlashInfer when they are available and falls back to PyTorch SDPA otherwise. Compiling cuRo