verl-project / verl

👋 Hi, everyone!
 verl is a RL training library initiated by ByteDance Seed team and maintained by the verl community.
 verl: Volcano Engine Reinforcement Learning for LLMs 
verl is a flexible, efficient and production-ready RL training library for large language models (LLMs).
verl is the open-source version of **HybridFlow: A Flexible and Efficient RLHF Framework** paper.
verl is flexible and easy to use with:
- **Easy extension of diverse RL algorithms**: The hybrid-controller programming model enables flexible representation and efficient execution of complex post-training dataflows. Build RL dataflows such as GRPO, PPO in a few lines of code.
- **Seamless integration of existing LLM infra with modular APIs**: Decouples computation and data dependencies, enabling seamless integration with existing LLM frameworks, such as FSDP, Megatron-LM, vLLM, SGLang, etc
- **Flexible device mapping**: Supports various placement of models onto different sets of GPUs for efficient resource utilization and scalability across different cluster sizes.
- Ready integration with popular HuggingFace models
verl is fast with:
- **State-of-the-art throughput**: SOTA LLM training and inference engine integrations and SOTA RL throughput.
- **Efficient actor model resharding with 3D-HybridEngine**: Eliminates memory redundancy and significantly reduces communication overhead during transitions between training and generation phases.
## News
- [2026/08] verl-vla `v0.1.0` is released: a unified VLA post-training framework for human-in-the-loop data collection, supervised fine-tuning, and reinforcement learning across simulators, real robots, and distributed cloud-edge resources, built on top of verl.
- [2026/08] VeRL-Tinker is released: keep the Tinker Cookbook loop you know, and run SFT, RL, and distillation on verl-managed GPU workers you control; read the blog here.
- [2026/08] VeRL-Omni `v0.2.0` is released: faster diffusion RL, rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more.
- [2026/07] RL-Insight is released: online observability for reinforcement learning training. RL-Insight connects training-side metrics, RL state traces, and service dashboards across distributed rollout and optimization workloads.
- [2026/06] verl-SpeCo is pre-released: a co-training framework for speculative decoding across RL training and inference, keeping draft models aligned during training and reusable for accelerated serving, built on top of verl.
- [2026/05] uni-agent is released: a unified agent framework to build, run, and train LLM agents at scale, built on top of verl.
- [2026/05] VeRL-Omni is pre-released: a unified RL stack for diffusion and omni-modal model post-training built on top of verl. Read the blog post for details.
- [2026/05] verl's zero-mismatch HuggingFace rollout vexact is released: with batch-invariant kernels, shared model definition with FSDP, and out-of-box examples compatible with VeOmni.
- [2026/04] verl's Mega