chenkangjie1123 / VGGT-Diff

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis 
 Kangjie Chen 1* ,  Xiangyu Li 1* ,  Dongbin Zhang 1 ,  Chaoda Zheng 1 ,  Shijia Chen 1 ,  Jinhao Deng 1 ,  Hongbin Lin 2 ,  Choo Sin Wai 3  
 Minqi Wang 2 ,  Minghao Yang 3 ,  Dake Zhong 3 ,  Guorui Song 3 ,  Yu Zhang 1 ,  Xianming Liu 1 ,  Boyang Wang 1† 
 1 XPeng Motors    2 The Chinese University of Hong Kong    3 Tsinghua University 
 VGGT-Diff delivers strong novel-view synthesis across challenging in-domain and cross-domain viewpoints, outperforming existing generative NVS methods. Core model weights are now available on Hugging Face. 
## News
- 🎉 **Oct. 15, 2026: Stronger full-resolution NVS checkpoint released!** Our new 480x832 scaling checkpoint, trained with 100K half-resolution steps followed by 6K full-resolution steps, is now available on Hugging Face. 🎉
- 🎉 **Oct. 5, 2026: Model weights released!** All three core checkpoints are now on Hugging Face. 🎉
- **Sep. 29, 2026:** arXiv paper and pose-free inference released.
- **Sep. 27, 2026:** Paper, project page, and code released.
## Model Zoo
| Model | Task | Release setting | Download |
| --- | --- | --- | --- |
| **VGGT-Diff NVS 192x336 (100K)** | Independent-view NVS | DL3DV clean-10K; 100K effective steps | Model card and weights |
| **VGGT-Diff NVS 480x832 (100K + 6K)** | Full-resolution independent-view NVS | DL3DV clean-10K; 100K half-resolution steps + 6K full-resolution steps | Model card and weights |
| **VGGT-Diff NVS 480x832** | Full-resolution NVS used for the paper's baseline comparison | DL3DV clean-1K; ~18K half-resolution steps + 7,380 full-resolution steps | Model card and weights |
| **VGGT-Diff Trajectory 480x832 (36K)** | 80-frame continuous camera trajectories | Full-resolution trajectory adaptation | Model card and weights |
The repositories contain the trained VGGT-Diff DiT and conditioning weights. Wan2.1, VGGT-Omega, and other external model components remain separate dependencies and are not redistributed with these checkpoints.
Download a checkpoint with the Hugging Face CLI, for example:
 [code] 
Use `--training-mode nvs` with either NVS checkpoint. The trajectory checkpoint is the default model family for `scripts/infer.py` and `scripts/infer_pose_free.py`.
## Installation
 [code] 
Request access to the VGGT-Omega checkpoint and download `vggt_omega_1b_512.pt`. Wan2.1 weights are downloaded automatically on first use, or can be supplied through `--base-model-dir`. A local model root should contain `Wan-AI/Wan2.1-I2V-14B-480P/` and `Wan-AI/Wan2.1-T2V-1.3B/google/umt5-xxl/`; pointing to the `Wan2.1-I2V-14B-480P` directory itself is also supported when the tokenizer is available in the adjacent `tokenizer/` tree.
## Inference
The bundled example uses six source views, an 80-frame camera trajectory, and the released full-resolution trajectory checkpoint:
 [code] 
Outputs are saved to `outputs/garden/`. The model predicts 80 frames once and keeps those raw PNGs in `frames/`; `vgg