vanzll / DiffusionART

Efficient, stable and scalable reinforcement of diffusion models 
 Text-to-image  ·  Text-to-video  ·  Image-to-video 
 Method  / 
 Quick Start  / 
 FAQ  / 
 Documentation 
## News
- 🎉 **[2026.10.12]** Our paper, code, models and project page are now public! DiffusionART and six baselines share one framework for **T2I, T2V and I2V**. Explore the training recipes!
 Changelog 
- **2026.10.17:** ART now splits each prompt's trajectories evenly between its
 two update windows, retaining all prompts in each update even when a rank
 handles multiple prompts. Guidance still uses the full K-sample group.
 Image training uses a stratified random split; video training preserves
 trajectory order. Baseline update schedules are unchanged.
- **2026.10.14:** Fixed Hunyuan paper recipes that defaulted to rollout/update
 microbatches of 1 instead of 8 (ReFL remains 1/1). Microbatch size can strongly
 affect GPU utilization and training throughput. Use
 commit `0fb23a7`
 or a later version containing this fix. Smaller microbatches remain supported
 for memory-constrained devices; see batch-size guidance.
---
**DiffusionART** is an on-policy diffusion RL algorithm via **Aggregation-First Reward
Tilting with Adaptive Restoration**. It combines the advantages of forward-process and
reverse-process reinforcement learning while avoiding their limitations.
DiffusionART is **highly efficient, stable and simple**, requires minimal hyperparameter
tuning and no differentiable reward model, naturally supports pairwise
feedback, and is robust to small rollout groups for scalable diffusion reinforcement learning.
This repository provides a comprehensive framework for fine-tuning visual
generation models, supporting widely used open-source **base models, reward
models and state-of-the-art algorithms** across text-to-image, text-to-video
and image-to-video generation.
Fine-tuning HunyuanVideo (13B) (left) and
 Wan2.1-I2V-14B-480P, CFG/step-distilled (14B) (right)
using VideoReward , with Visual Quality (VQ):Motion Quality (MQ) = 1:1 .
Mean train reward is the mean of raw VQ and MQ. T2V ratios compare GPU hours to reach each baseline's final reward level.
## Method
### Motivations and Insights
1. NFT-style forward-process RL contains a hidden **forward-reverse
 consistency force** in its objective, slowing and destabilizing training.
2. Off-policy strategies stabilize reinforcement learning of continuous-space generative
 models through a **strong restoration force**. Even a small output-space
 restoration MSE can produce a large aggregated parameter gradient.
### Aggregation-First Guidance and Adaptive Restoration
DiffusionART is simple: roll out **K samples per prompt** with an ODE solver,
compute the reward-endpoint covariance as **guidance**, apply it to the visited
states and actions, then perform **two disjoint minibatch updates**.
We call this reward-improving direction **guidance**, bringing
a classifier-free guidance (CFG)-like idea into diffusion RL: the model learns
t