Robbyant / lingbot-map

LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction 
Robbyant Team
https://github.com/user-attachments/assets/fe39e095-af2c-4ec9-b68d-a8ba97e505ab
-----
### 🗺️ Meet LingBot-Map! We've built a feed-forward 3D foundation model for streaming 3D reconstruction! 🏗️🌍
LingBot-Map has focused on:
- **Geometric Context Transformer**: Architecturally unifies coordinate grounding, dense geometric cues, and long-range drift correction within a single streaming framework through anchor context, pose-reference window, and trajectory memory.
- **High-Efficiency Streaming Inference**: A feed-forward architecture with paged KV cache attention, enabling stable inference at ~20 FPS on 518×378 resolution over long sequences exceeding 10,000 frames.
- **State-of-the-Art Reconstruction**: Superior performance on diverse benchmarks compared to both existing streaming and iterative optimization-based approaches.
---
## 📑 Table of Contents
 Click to expand 
- 📰 News
- 📋 TODO
- ⚙️ Installation
- 📦 Model Download
- 🚀 Quick Start
- 🎬 Interactive Demo (`demo.py`)
 - Try the Example Scenes
 - Streaming with Keyframe Interval
 - Windowed Inference (for long sequences, >3000 frames)
 - Sky Masking
 - Visualization Options
 - Performance & Memory
- 🎥 Offline Rendering Pipeline (`demo_render/batch_demo.py`)
- 📜 License
- 📖 Citation
- ✨ Acknowledgments
---
## 📰 News
- **2026-06-28** — Fixed an SDPA KV cache bug. **The SDPA backend now performs better on long sequences**. We still recommend the FlashInfer backend for the best performance.
- **2026-05-25** — 📊 **Evaluation benchmark released**. We released the evaluation scripts for KITTI and Oxford Spires — see benchmark/ for the pipeline, and run `preprocess/oxford.py` to prepare Oxford Spires data before evaluation.
- **2026-04-29** — 📹 **Long-video demo released**. We released a very-long-video example (~25 000 frames, 13-minute indoor walkthrough) rendered with the offline pipeline — see Worked Example for the command, flag rationale, and rendered output.
- **2026-04-27** — 🚀 **LingBot-Map accelerated**. Pull the latest `main` and run `python demo.py --compile ...` or `python gct_profile.py --backend flashinfer --dtype bf16 --compile` to verify on your hardware.
- **2026-04-24** — Fixed a FlashInfer KV cache bug where `--keyframe_interval > 1` silently cached non-keyframes. **You should now see better pose and reconstruction quality when running with more than 320 frames**.
---
## 📋 TODO
- ✅ Release evaluation benchmark
 - ✅ Oxford Spires dataset
 - ✅ KITTI dataset
 - ✅ VBR dataset
 - ✅ Droid-W dataset
 - ✅ TUM-D dataset
 - ✅ 7-scenes dataset
 - ✅ ETH3D dataset
 - ✅ Tanks and Temples dataset
 - ✅ NRGBD dataset
- ✅ Release demo scripts
 - ✅ Indoor long-video demo (Featured indoor walkthrough)
 - ✅ Outdoor long-video demo
 - ✅ LingBot-World demo (Worked example)
 - ✅ Aerial long-video demo
---
## ⚙️ Installation
**1. Create conda environment**
 [code] 
**2. Install PyTorch (CUDA 12.8)**
 [code] 
> PyTorch 2