yiboz2001 / UltraTex

UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing
 SIGGRAPH Asia 2026 
 Yibo Zhang 1,2 ,
Ze Yuan 3 ,
Nan Cao 4,2 ,
Li Zhang 5,2 ,
Yan-Pei Cao 6 ,
Yuan-Chen Guo 6 ,
Rui Ma 1 † 
 1 Jilin University  
 2 Shanghai Innovation Institute  
 3 The University of Hong Kong
 4 Tongji University  
 5 Fudan University  
 6 VAST
 † Corresponding author
**UltraTex** is an efficient end-to-end framework for high-resolution multi-view diffusion-based 3D texturing at **2048×2048**. Object-centric multi-view renderings contain two major sources of redundancy: background-induced sequence redundancy and sparse token interactions within the foreground. UltraTex addresses them with:
1. **Background Token Dropping (BTD)** — discard background tokens before the DiT backbone, keeping original RoPE indices so spatial coordinates and the multi-view geometric prior survive.
2. **Block-Sparse Attention (BSA)** — top-*k* sparse attention over the retained foreground sequence.
3. **Foreground-Aware VAE Decoding** — replace the undenoised background latent with a canonical in-distribution background and lightly fine-tune the decoder so 2K reconstruction stays artifact-free.
On common samples in G-buffer TexVerse, this yields **20.6×–91.1×** training speedup and **22.3×–74.6×** end-to-end inference speedup over the dense baseline.
Feel free to contact me (ybzhang23@mails.jlu.edu.cn) or open an issue if you have any questions or suggestions.
## News
- [2026-10-03] Training and inference code and weights released.
- [2026-09-22] Paper available on arXiv.
- [2026-09-22] Project page and G-buffer TexVerse released.
## Installation
Tested with Python 3.12, CUDA 12.4 and PyTorch 2.5.1.
 [code] 
`requirements.txt` pins the exact versions we tested. The sparse-attention
kernels are Triton (`triton==3.1.0`) and are compiled on first use.
Download the UltraTex weights into `checkpoints/` (layout below):
 [code] 
## Repository structure
 [code] 
## Weights
UltraTex ships weights for two backbones. Each needs its own LoRA **and** its
own Foreground-Aware VAE decoder — the decoders are not interchangeable, since
the two backbones use different autoencoders.
Download the UltraTex weights from ModelScope and place them under `checkpoints/`:
 [code] 
Base models come from their original sources:
FLUX.1-dev,
FLUX.2,
Qwen3-8B, and the
XLabs T5 /
CLIP encoders for FLUX.1.
A LoRA only loads into the backbone it was trained on: the 9B LoRA will not fit
the 4B model (hidden size 4096 vs 3072).
## Training
FLUX.1-dev backbone:
 [code] 
FLUX.2-Klein 9B backbone (albedo, or the metallic-roughness branch with `TASK=mr`):
 [code] 
Both default to the bundled demo objects (`data/train_demo.json`). Training
continues from the released LoRA when it is present under `checkpoints/`,
otherwise the LoRA is trained from scratch.
Key environment variables (all overridable, see scripts for defaults):
| Variable | Description |
|---|---|
| `TRAIN_JSON` | Path to training data JSON |
| `EVAL_JSON` | Path t