Yzmblog / DMAD

# DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
### 4-step MiniMax-H3 students for joint audio-video generation
Zhengming Yu 1,2 ,
Junkun Yuan 2 ,
Haotian Yang 2 ,
Gordon Guocheng Qian 2 ,
Yizhi Wang 2 ,
Angtian Wang 2 ,
Yiding Yang 2 ,
Bo Liu 2 ,
Xin Li 1 ,
Wenping Wang 1 ,
Chongyang Ma 2 
 1 Texas A&M University, 2 ByteDance 
This repository contains the **4-step DMAD students of MiniMax-H3
(33B, text-to-audio-video)** and the code to run them: a LoRA of rank 128 on the H3 transformer turns the 50-step
teacher into a 4-step generator of 1344x768 video with native stereo audio.
## 🗣️ Updates
* **2026/10/15:** The Wan2.1 training data (42,158 Wan2.1-14B
 teacher samples with their real-video latents) is on Hugging Face, so `train/wan` can start
 training right away 🤗
* **2026/10/10:** 8 GB GPUs: `--low-vram` now makes 5 s videos in under 7 GiB of GPU memory and
 15 s videos in 12 GiB, bit-identical to the larger-GPU runs (`--low-vram 8 / 12 / 16 / 24` picks the settings for
 your GPU, detected by default), and a 5 s ComfyUI workflow for 8 GB GPUs 🪶
* **2026/10/08:** 16 GB GPUs: `--low-vram` now also makes 15 s videos, and both it and the
 ComfyUI workflows fit in 14 GiB of GPU memory with output
 bit-identical to the larger-GPU runs 🪶
* **2026/10/07:** Ready-to-run ComfyUI workflows (4 and 8 steps,
 15 s of video with audio) that run on a 24 GB GPU 🎛️
* **2026/10/07:** `--low-vram` now also works on Windows, thanks to @hiroki-abe-58 (#6)
* **2026/10/06:** The DMAD weights of Wan2.1 (1.3B, 14B), SDXL and ImageNet-64 and the
 MiniMax-H3 training data are on Hugging Face 🤗
* **2026/10/05:** Training code for ImageNet-64, SDXL, Wan2.1 and MiniMax-H3, and ComfyUI support 🏋️
* **2026/10/04:** `--low-vram` inference on a 24 GB consumer GPU ⚡️
* **2026/10/02:** Inference code and the 4-step MiniMax-H3 students released 🚀
## 📝 Abstract
> Distribution Matching Distillation (DMD) trains a few-step student on the difference between target and student
> scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory
> and computation cost. We introduce DMAD, **Distribution Matching as Adversarial Distillation**, which recasts
> distribution matching as classification and learns the required log-density ratios directly. Two discriminator
> heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their
> logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses
> recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator
> logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across
> noise levels from the real-data head's empirical logit gap between real and teacher samples. DMAD reaches a Fréchet
> Inception Distance (FID) of 1.04 with one-step generation on Ima