sgl-project / sglang

# SGLang: Fast inference for LLMs and multimodal models
 Docs |
 Cookbook |
 Website |
 Blog |
 Slack 
SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the `sglang` Python package.
👋 Get started below, or meet the community at SGLang Events, including meetups, developer meetings, workshops, and office hours.
> **SGLang Summit 2026: Meet the builders of open AI** 
> **Our flagship annual gathering · November 12–13 · San Francisco**
>
> Go deep on inference, training, and AI hardware through technical talks, live demos, and hands-on workshops. Bring your deployment challenges and meet the engineers and researchers building the stack.
## Get Started
Pull the Docker image, which includes SGLang and its dependencies:
 [code] 
Alternatively, install SGLang in an activated Python environment with uv:
 [code] 
Next, launch your model:
- Quickstart: Run your first model and send a request.
- Cookbook: Choose your model and hardware to get a ready-to-run launch command.
## Supported Hardware
SGLang supports a wide range of GPUs, TPUs, NPUs, CPUs, and Apple Silicon platforms.
| Platform | Representative hardware |
| --- | --- |
| NVIDIA | B200/B300/GB200/GB300; H100/H200/H800/H20; A100; RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin |
| AMD | Instinct MI300X, MI325X, MI350X, MI355X |
| Google TPU | v6e, v7; SGLang-TorchTPU / SGL-JAX |
| Intel (GPU / CPU) | Arc / Arc Pro B-Series GPUs, Xeon CPUs |
| Apple Silicon | Macs via Metal / MLX |
| Huawei Ascend | A2, A3, 950PR/DT NPUs |
| Moore Threads | MTT S5000 GPUs |
Integrations in progress: AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU, Iluvatar CoreX, and more.
SGLang-TorchTPU brings the SGLang serving engine to Google TPUs through PyTorch, torch_tpu, and Pallas kernels. It supports LLM and vision-language serving on TPU7x (Ironwood), including tensor and expert parallelism across hosts. SGL-JAX provides the JAX-based TPU serving path.
See the Cookbook and platform guides for model compatibility and setup.
## SGL Ecosystem
| Area | Projects | Purpose |
| --- | --- | --- |
| Diffusion | SGLang Diffusion | Built into SGLang for image and video generation with diffusion models. |
| Audio | SGLang Omni | Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR). |
| Speculative Decoding | SpecForge | Train draft models for speculative decoding and deploy them with SGLang. |
| RL and Post-Training | Miles, slime, AReaL, Tunix, verl | Training frameworks that integrate SGLang for rollout generation. |
| Deployment and Orchestration | SMG, RBG, llm-d, Ray Serve, NVIDIA Dynamo | Deploy and scale SGLang inference services with routing, load balancing, and cluster orchestration. |
| Education | Mini-SGLang, zero