huggingface / accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

NVIDIA-NeMo / labs-molt

A scalable, agentic-first, and HuggingFace-native RL framework for research (9k lines).

sgl-project / SpecForge

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

yandex / YaFSDP

YaFSDP: Yet another Fully Sharded Data Parallel

zengxiao-he / tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

meta-pytorch / torchft

Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)