VAST-AI-Research / EdenLive

# EdenLive
**EdenLive: Real-Time Generative Rendering from Any Visual Reference**
Project Page · Code · Models · Paper
Zehuan Huang 1,\* , Tianxing Xu 1,\* , Chunli Peng 1,\* , Cheng Zhang 1,\* , Zeyi Zheng 1,† , Yifei Song 1,† , Zekai Gu 1,† , Zhipeng Yu 1 , Peng Wang 1,‡ , Yanpei Cao 1,✉ 
\* Equal contribution · † Intern at VAST · ‡ Project lead · ✉ Corresponding author
 1 VAST · Project Eden
**TL;DR:**
* For interactive world models, we decouple state evolution (maintained by coding agents or engines) and appearance (rendered by EdenLive).
* EdenLive renders reference videos to enhanced videos in real time at 24 fps or higher, even on a consumer GPU (RTX 5090).
https://github.com/user-attachments/assets/243f1392-b41e-4ee6-a9c0-f5be8263bbec
## Overview
EdenLive is a real-time generative renderer that treats the
reference video as the interface between an interactive program and neural visual synthesis: the upstream program executes the rules and maintains the world state, whereas EdenLive renders the resulting reference video, guided by a target first frame and a text prompt, into detailed frames.
The project page demonstrates over 24 FPS, minute-scale generation, and 480P/720P output. The 5B model supports single-GPU deployment at 720P.
We have open-sourced:
- [x] **5B and 14B model weights**, for both bidirectional and few-step (chunk-AR) inference.
- [x] **Real-time streaming inference code**.
- [x] **EdenGolem（v0.1） skill that turns a text prompt into a coarse playable game.**
- [x] **A frontend to try the full prompt-to-game pipeline.**
## Installation
One Python environment runs both the offline inference in `edenlive/` and the
real-time service in `realtime/`.
| | Verified version |
| --- | --- |
| OS / GPU | Linux x86-64, NVIDIA GPUs with 80 GB (CUDA 13 driver) |
| Python | 3.10 |
| PyTorch | `2.14.0.dev20260717+cu130` nightly (`realtime/requirements.txt` pins `2.13.0`, cu130), torchvision 0.28.0 |
| diffusers / transformers / accelerate | 0.39.0 / 5.12.1 / 1.0.1 |
| decord / pandas / pyarrow | 0.6.0 / 2.3.3 / 25.0.1 |
| imageio / imageio-ffmpeg | 2.37.4 / 0.6.0 |
| Node.js | 22 (only to fetch three.js for the game renders) |
 [code] 
The default 5B bidirectional profile uses FA4 temporal text
attention and requires SM90, BF16, `flash-attn-4==4.0.0b31`, and
`nvidia-cutlass-dsl==4.7.1`. Use `--no-deps` when installing kernels into an
existing environment to preserve PyTorch. See `docs/zh/wan22-fa4.md`.
## Model weights
 [code] 
Everything loads from one directory, `models/EdenLive` under the repository root by
default (about 210 GB in total).
 [code] 
The default weight directory is `models/EdenLive`, relative to the working directory.
To specify another directory:
 [code] 
The real-time service reads the same tree through `MODEL_ROOT` in
`realtime/deploy.conf`.
### Quick start
Two proxy recordings ship with the repo under `examples/samples/`, with a manifest that
points at them:
 [code] 
`recording.mp4` is the greybox proxy (1280x704),