NoizAI / YuE2-Turbo

English | 中文 
 YuE2-Turbo: Fast, Concurrent Inference for YuE2 
 Same YuE2 model. 1.68× faster per song. 3.31× more songs per GPU. 
 🧰 Environment ·
 🚀 Deploy ·
 ⚙️ Configuration ·
 🎵 About YuE2 ·
 🤗 Weights 
**YuE2-Turbo is an inference acceleration and serving layer for YuE2.** It keeps the released `YuE2-3B` weights and the standard generation recipe (BF16, 32 flow-matching steps) and replaces how they are executed: the autoregressive stages run on **vLLM**, all weights stay **resident on the GPU**, acoustic synthesis is **batched across requests**, and an **HTTP job API + browser Studio** turn one GPU into a concurrent song-generation service.
| Scenario | Original YuE2 | YuE2-Turbo | Speedup |
|---|---:|---:|---:|
| Single request (RTF, lower is better) | 0.290 | 0.173 | **1.68×** |
| 4 concurrent requests (system RTF) | 0.317 | 0.096 | **3.31×** |
*Measured on one NVIDIA RTX 5090 (32 GB), PyTorch 2.10.0 + CUDA 12.8, vLLM 0.19.0, BF16, 32 ODE steps, after warmup. RTF = seconds of compute per second of generated audio: at RTF 0.17, a 60-second song takes about 10 seconds. Single request: 3 songs × 3 repeats. Concurrent: the same 4 requests per wave, 3 waves, wall time ÷ total audio generated. Reproduce with `yue2-benchmark`.*
The accelerated path does not lose quality on the WildSongBench standard protocol. Each of 192 prompts is generated twice; the lower-PER candidate is scored against the published YuE2 row:
| Metric | Original YuE2 | YuE2-Turbo |
|---|---:|---:|
| SongBench Avg ↑ | 6.7316 | 6.7623 |
| MuLan ↑ | 0.5068 | 0.5069 |
| AllMusicCaps ↑ | 0.4054 | 0.4087 |
| PER ↓ | 8.44% | 8.18% |
*One RTX 5090, vLLM with 4-way concurrency, BF16, 32 ODE steps, same released weights and generation recipe.*
## Environment
| Requirement | Tested / recommended |
|---|---|
| OS | Linux x86_64 |
| GPU | NVIDIA with BF16 support. Defaults are tuned for a **32 GB RTX 5090**; on 24 GB cards lower `YUE2_AR_CONCURRENCY` / `YUE2_VLLM_MAX_NUM_SEQS` or set `YUE2_RESIDENT_MODELS=false` and validate |
| Driver | Supports CUDA 12.8 |
| Python | 3.11 or 3.12 |
| PyTorch | 2.10.0 (`cu128` wheels) |
| vLLM / Triton | 0.19.0 / 3.6.0 (installed by the `server` extra) |
| Disk | ~8 GB for `YuE2-3B` + `YuE2-Vae`, plus a few GB for the derived vLLM AR checkpoint cache |
GPU memory at rest with the default configuration is about 18 GB (PyTorch MoT + VAE ≈ 8 GB, vLLM engine ≈ 10 GB); peaks under 4-way load stay under 25 GB on a 32 GB card.
## Deploy the accelerated service
### 1. Install
 [code] 
The `server` extra pulls in FastAPI, Uvicorn, vLLM, and Triton. Weights download from Hugging Face on first start; set `YUE2_MODEL` / `YUE2_VAE` to local directories and `YUE2_LOCAL_FILES_ONLY=true` for offline hosts.
### 2. Configure
 [code] 
The defaults already select the accelerated path (`YUE2_BACKEND=vllm`, `YUE2_RESIDENT_MODELS=true`, `YUE2_AR_CONCURRENCY=4`, `YUE2_NAR_BATCH_SIZE=2`, `YUE2_AR_NAR_OVERLAP=true`). See the configuration reference to tune them.
### 3. Start th