HKUST-LongGroup / UniAVC

Unified Audio-Video Creation via Progressive Training and Asymmetric Scheduling 
**Kien T. Pham 1 , Yuhao Liu 2 , Chong Hou Choi 2 , Zhenqi He 1 , Yanghao Wang 1 , Chenyang Lei 2 , Rui Liu 2 , Qifeng Chen 1 , Long Chen 1 **
 1 The Hong Kong University of Science and Technology    2 Huawei Research
 tkpham@connect.ust.hk  ·  longchen@ust.hk 
---
## 🎬 Demo
---
## Overview
UniAVC is a unified model that natively handles text-conditioned joint audio-video generation, bidirectional cross-modal synthesis, and instruction-guided editing of either or both modalities without relying on task-specific adapters. To achieve this versatility, the model utilizes a progressive training paradigm that first masters unimodal generation and editing capabilities before dedicating scarce paired datasets entirely to establishing cross-modal correspondence and temporal synchronization. The framework further introduces an asymmetric noise-level scheduling technique that derives decoupled per-branch noise trajectories and explicitly supervises extreme boundary states to enable intent-aware editing and post-inference self-refinement. Extensive empirical evaluations demonstrate that this unified approach delivers performance comparable to specialized, state-of-the-art baselines across a wide range of generation and editing benchmarks.
## Highlights
### Comparison with the state of the art
One sample per editing family, applying the same instruction with UniAVC and with
InstructAV2AV and MiniMax-H3.
#### Subject Editing
> Transform the man into a cyberpunk painting style with high-saturation neon colors and sharp digital illustration strokes, while keeping all characters, actions, and composition unchanged.
 Input 
 UniAVC 
 InstructAV2AV 
 MiniMax-H3 
#### Subject Addition
> Add a person wearing white clothes, standing still in the center of the frame, looking at the camera with a blank expression.
 Input 
 UniAVC 
 InstructAV2AV 
 MiniMax-H3 
#### Subject Removal
> Remove the woman wearing a white lab coat, a stethoscope, and looking confused or distressed from the source video, while preserving the background of a blurred office environment, a desk lamp, the wall calendar, and the shelving behind her.
 Input 
 UniAVC 
 InstructAV2AV 
 MiniMax-H3 
#### Speech Editing
> Alter the character's dialogue to: "We were in the drum circle like, wow, this is where real musical magic happens right now!" while keeping visual elements and background sounds identical to the original scene.
 Input 
 UniAVC 
 InstructAV2AV 
 MiniMax-H3 
#### Background Editing
> Replace the background with a private study at late night, with quiet city night lights visible outside the window, while keeping the person's image, clothing, actions, and contemplative head-down expression in the foreground completely consistent.
 Input 
 UniAVC 
 InstructAV2AV 
 MiniMax-H3 
### Editing versatility
Each grid holds the source clip and the results of three different instructions applied to that same clip.
#### Su