baidu-baige / LoongForge

English | 简体中文 
 Train LLMs, VLMs, diffusion, and embodied models — faster. 
 Docs 
  · 
 Blog 
  · 
 Performance 
  · 
 Quick Start 
  · 
 Supported Models 
  · 
 Awesome Projects 
**LoongForge** is an open-source training framework developed by the Baidu AI Cloud Baige team, built to deliver faster training for mainstream **LLMs, VLMs, diffusion, and embodied models**.
- **High Performance** — Built on Megatron-LM and torch-native backends, with deep optimizations for training speed that keep **training loss aligned with the baseline**.
- **Easy to Use** — Ready-to-run configs and launch scripts for 40+ open-source models, plus checkpoint conversion and data preprocessing tools. Supports **pre-training**, **mid-training**, **SFT**, and **LoRA**.
- **Production-Proven** — Open-sourced from AIAK-Training-LLM, which powers enterprise customers' proprietary models and several open-source projects. Production runs reach **5,000+ XPUs**.
---
**Example:** training the embodied model DreamZero on LoongForge reaches 4.38× the open-source baseline's throughput, with loss curves staying aligned
 ⭐ Give us a Star to help more people discover LoongForge and grow the community. 
## 🏗️ Architecture
Since optimal training strategies differ across model families and scales, LoongForge adopts a multi-backend architecture.
- **Megatron Backend** — For LLM, VLM, and diffusion models, extended with MoE parallelism, per-component heterogeneous parallelism, long-sequence optimizations, etc.
- **torch-native Backend** — For embodied models (VLA and WAM). Built on native PyTorch with **DDP / ZeRO-1 / FSDP / HSDP**, with deep optimizations for representative models across I/O, communication strategy, kernel efficiency, etc.
## 🔥 Latest News
- **[2026/10]** 🤖 Introducing **LoongForge-VLDA**, a Vision-Language-Delta-Action model trained with LoongForge: **98.5%** on LIBERO, **84.5%** zero-shot on LIBERO-Plus, and **+17–18 points** over π0.5 on real robots. [project page] [paper]
- **[2026/10]** 🦾 Open-sourced **ego2robot** — a pipeline that converts first-person videos of human manipulation into dual-arm robot training data, outputting **LeRobot v3.0** format across **16 robot morphologies**.
- **[2026/09]** ✨ Added training support for **GLM-5.3-flash**.
- **[2026/09]** ✨ Added **Kimi-K3** BF16 training support for both LLMs and VLMs.
- **[2026/09]** ⚡ Added an optimized **DreamZero Wan2.2-5B FSDP recipe** with cache-aware data loading, compiled attention blocks, frozen-module handling, and FSDP2 Delta-FP8 Param AllGather.
- **[2026/08]** 🤖 Added VLA training support for **Wall-OSS-0.5**, with custom fused operators for higher training throughput.
- **[2026/08]** 📄 Released the **TAOT paper** — topology-aware dynamic expert replica placement that tackles expert-parallel (**EP**) load imbalance in **MoE** training, cutting overhead by up to **74%** over industry solutions, with **1.43× speedup** measured on a real training case. [blog]
- **[2026/08]** ✨ Added training sup