ifradlin / arrow

ARROW
 Arbitrary Reconstruction and Tracking of 4D Observations in the Wild 
 Ilya Fradlin   
 Christian Schmidt   
 Jens Piekenbrinck 
 Karim Knaebel   
 Gonzalo Martin Garcia   
 Bastian Leibe 
 RWTH Aachen University 
## Updates
**Coming next:** evaluation and training code.
- **2026-10-12:** Inference code, pretrained weights, and an interactive viewer are released.
- **2026-10-01:** Our paper is available on arXiv.
## Overview
**ARROW** is a feed-forward model that unifies **3D reconstruction and 3D point tracking from arbitrary image sets**. It combines the flexible point-query decoding of D4RT with support for arbitrary observations across viewpoints and capture times. This enables reconstruction and tracking from moving-camera videos, multiple video streams, and unordered photo collections.
ARROW achieves state-of-the-art 3D tracking on WorldTrack and TAPVid-3D and outperforms dedicated multi-view trackers on an adapted **RGB-only MVTracker benchmark**, remaining competitive on 3D reconstruction tasks.
Visit the project page for interactive examples and results.
## Installation
Clone the repository and install with uv:
 [code] 
This installs ARROW into `.venv` in editable mode.
On Linux and Windows, `uv sync` installs PyTorch built for CUDA 13.0 (needs a recent NVIDIA driver).
On macOS it uses Apple Silicon GPUs (MPS).
ARROW picks CUDA, MPS, or CPU automatically. To choose a device, pass `--device cpu`
(or `cuda:1`, `mps`) to `arrow infer` or `arrow-viewer`, or `device=` to `ARROW.from_pretrained()`.
For the interactive viewer, add `--extra viewer`; for development
(tests, linting, type checking), add `--group dev`.
## Viewer
Run inference and view 3D reconstructions, point tracks, depth, and camera
poses directly in the interactive viewer:
 [code] 
Open **http://127.0.0.1:7860** to select input images or videos, choose tasks, and
adjust run parameters such as frame selection and query spacing. Run inference
and visualize the results directly in the viewer. See the viewer guide
for input selection, run controls, layers, coloring, and filters.
## Usage
Alternatively, track points from the CLI and use the viewer to visualize the saved
predictions. This example takes every third video frame and uses the remaining
defaults from configs/inference.py:
 [code] 
The command saves `results/railway.npz` with the prepared input images, every
query (pixel, source, target, and reference frame), its predictions (3D points,
confidence, visibility, normals, ...), and metadata describing the frames and
tasks. View the saved results in the viewer:
 [code] 
This opens the saved predictions as a scene at **http://127.0.0.1:7860**.
See all options for other tasks,
grid spacing, and frame selection.
The same run from Python:
 [code] 
Weights download on first use. A local model directory also works; pass it to
`from_pretrained()` or use `--checkpoint /path/to/model`.
See inference examples for selected-point tracking,
multi-view input, poses, and saved predict