The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.
Auto detecting, masking and inpainting with detection model.
OneTrainer is a one-stop solution for all your Diffusion training needs.
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.
Fast stable diffusion on CPU and AI PC
A pipeline parallel training script for diffusion models.
[ICCV 2025 Highlight] OminiControl: Minimal and Universal Control for Diffusion Transformer
PALLAIDIUM — a generative AI movie studio, seamlessly integrated into the Blender Video Editor (VSE), enabling end-to-end production from script to screen and back.
Scalable and memory-optimized training of diffusion models
A full-featured image/video management app with AI-powered organization and semantic search. Supports metadata from SD-webui, ComfyUI, Fooocus, NovelAI, StableSwarmUI, and more. Available as standalone app, SD-webui extension, or library.
Pruna is a model optimization framework built for developers, enabling you to deliver faster, more efficient models with minimal overhead.
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
Official code of Motus: A Unified Latent Action World Model
Examples of programs built using Modal
[NeurIPS2026] We present StableAvatar, the first end-to-end video diffusion transformer, which synthesizes infinite-length high-quality audio-driven avatar videos without any post-processing, conditioned on a reference image and audio.
Nodes for better inpainting with ComfyUI: Fooocus inpaint model for SDXL, LaMa, MAT, and various other tools for pre-filling inpaint & outpaint areas.
a self-hosted webui for 30+ generative ai
A foundation model that generates synchronized video and audio in a single model
[ECCV 2026] Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery
Official implementation of ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation (SIGGRAPH 2026).
[ICML 2026] Official codebase for "Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation" & Causal Forcing++
Tiny AutoEncoder for Stable Diffusion (and other image models)
Unified Codebase for Advanced World Models.
DIPY is the paragon 3D/4D+ medical imaging library in Python. Contains generic methods for spatial normalization, signal processing, machine learning, statistical analysis and visualization of medical images. Additionally, it contains specialized methods for computational anatomy including diffusion, perfusion and structural imaging.
DeepInverse: a PyTorch library for solving imaging inverse problems using deep learning
Open data and scalable training for long-horizon video world models.
An easy to understand TTS / SVS / SVC framework
ComfyUI as a serverless API on Runpod
Open, Jev-compatible System One decision server on DiffusionGemma
Nodes for using ComfyUI as a backend for external tools. Send and receive images directly without filesystem upload/download.
🎨ComfyUI standalone pack with 40+ custom nodes. | ComfyUI 大号整合包,预装大量自定义节点(不含SD模型)
Bilingual (中文+EN) ML / LLM / diffusion / agent interview cheat sheets for AI 秋招 — generated by ARIS /interview-cheatsheet, rendered by /render-html into single-file HTML, reads anywhere — plus a CV→DBLP-fact-checked academic homepage generator and hand-authored long-form blogs 🌱
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention
[ICCV 2025 & TPAMI] VistaDream: Sampling multiview consistent images for single-view scene reconstruction
本地优先的全栈 AI 创作工作站:AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块;vLLM / llama.cpp / Transformers 多引擎本地推理,显存智能调度、舱壁隔离自愈、仅供学习使用,欢迎交流指正。(最新进度(除插件系统)受到协议限制,暂时无法提交到仓库)
Visionary: The World Model Carrier Built on WebGPU-Powered Gaussian Splatting Platform
RESTai is an AIaaS (AI as a Service) open-source platform. Supports many public and local LLM suported by Ollama/vLLM/etc. Precise embeddings usage, tuning, analytics etc. Built-in image/audio generation with dynamic loading generators. Live chat deployment. Built-in block based graphical language. Prompt versioning and much more...
[ICML'26] PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation
SCoPE: Sightline-Coordinate Positional Encoding for 3D-Aware Video Generation
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
AgentGarten: Code Worlds for Evolving Agents
Structured decisions with DiffusionGemma and vLLM. Native image inputs, image choices, live camera sampling, and a compact one-step inference path.
Veda Sparse Attention for ComfyUI
NVIDIA Sol-Attn for ComfyUI / Triton kernel on SM89 - SM121, with zero-copy MiniMax H3 nodes: memory-efficient attention, scheduled tau with graph preview, and feed-forward chunking. Measured 1.14–1.44× vs SageAttention and −37% MLP peak VRAM on H3
Build a person once, then shoot a whole series: framing, pose, placement, expression and aspect ratio vary while the person stays the same. ComfyUI nodes.
[ECCV 2026] Official implementation of "HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models"
Official code repository for "PixRestore: Unified Image Restoration via Pixel Diffusion Transformer".
[COLMW'26 Spotlight] AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Jev-style structured decisions on DiffusionGemma: the example server from vLLM PR 57250
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Hierarchical AR-to-Diffusion Distillation for Efficient Autonomous Driving VLA
Train LoRAs for Qwen-Image 2.1, FLUX.2 Klein 9B, FLUX.3 Image Klein, Krea 2, Z-Image, MOSS-Image, ERNIE-Image, Ideogram 4, Anima, SDXL (Pony, Illustrious, NoobAI), LTX 2.3, MiniMax-H3, and more on NVIDIA GPUs from 4–8 GB VRAM. One install, one launcher, NF4 models. Windows, Linux and RunPod.
基于 Tree-Ring 水印工程,实现潜空间 LWE 环形隐写与 NC 检测,支持 Stable Diffusion 2.1 实验、信号失真和几何攻击、频率/小波能量分析,支持gradio Web界面实验。
Official implementation of the paper "VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis."
Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusion
MLX (Apple Silicon) port of MiniMax-H3 — 33B joint video+audio diffusion. Validated against the diffusers reference; AdaLN precompute drops 13B at inference.
InternLumina-U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding, Image Generation and Editing
Official Implementation for Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Predictive Action Diffusion for Steerable Onboard Humanoid Control