PaddlePaddle / FlashMask

Efficient Mask-Aware Distributed Attention for Long-Context Training

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

deepspeedai / DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible

huggingface / pytorch-image-models

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

celery / celery

Distributed Task Queue (development branch)

Vonng / ddia

《Designing Data-Intensive Application》DDIA 第一版 / 第二版 中文翻译

optuna / optuna

A hyperparameter optimization framework

skypilot-org / skypilot

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

Netflix / metaflow

Build, Manage and Deploy AI/ML Systems

huggingface / accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

NVIDIA / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

algorithmicsuperintelligence / openevolve

Open-source implementation of AlphaEvolve

hyperopt / hyperopt

Distributed Asynchronous Hyperparameter Optimization in Python

netalertx / NetAlertX

Centralized network visibility and continuous asset discovery. Monitor devices, detect change, and stay aware across distributed networks.

mher / flower

Real-time monitor and web admin for Celery distributed task queue

yeasy / blockchain_guide

Introduce technologies on blockchain and distributed ledger, from theory to practice with bitcoin, ethereum and hyperledger.

bojieli / ai-infra-book

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

rllm-org / rllm

Democratizing Reinforcement Learning for LLMs

gpustack / gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

ag2ai / faststream

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

Bogdanp / dramatiq

A fast and reliable background task processing library for Python 3.

llm-d / llm-d

Achieve state of the art inference performance with modern accelerators on Kubernetes

astroautomata / PySR

Symbolic Learning in Python and Julia

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

PaddlePaddle / PARL

A high-performance distributed training framework for Reinforcement Learning

tensorflow / lingvo

Lingvo

open-telemetry / opentelemetry-python

OpenTelemetry Python API and SDK

EMI-Group / evox

Distributed GPU-Accelerated Framework for Evolutionary Computation. Comprehensive Library of Evolutionary Algorithms & Benchmark Problems.

mitogen-hq / mitogen

Distributed self-replicating programs in Python

Lightning-AI / torchmetrics

Machine learning metrics for distributed, scalable PyTorch applications.

taskiq-python / taskiq

Distributed task queue with full async support

ByteDance-Seed / VeOmni

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

cfregly / ai-performance-engineering

Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and full-stack tuning.

pytest-dev / pytest-xdist

pytest plugin for distributed testing and loop-on-failures testing modes.

faust-streaming / faust

Python Stream Processing. A Faust fork

ceph / ceph-ansible

Ansible playbooks to deploy Ceph, the distributed filesystem.

Yelp / paasta

An open, distributed platform as a service

dask / distributed

A distributed task scheduler for Dask

pyeventsourcing / eventsourcing

A library for event sourcing in Python.

intelligent-machine-learning / dlrover

DLRover: An Automatic Distributed Deep Learning System

google / vizier

Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.

Haidra-Org / AI-Horde

A crowdsourced distributed cluster for AI art and text generation

sgl-project / sglang-omni

SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.

modal-labs / modal-examples

Examples of programs built using Modal

brainix / pottery

Redis for humans. 🌎🌍🌏

nnaisense / evotorch

Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.

tiangolo / dockerswarm.rocks

Docker Swarm mode rocks! Ideas, tools and recipes. Get a production-ready, distributed, HTTPS served, cluster in minutes, not weeks.

NVIDIA-NeMo / Automodel

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

SandAI-org / MagiAttention

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

AgileRL / AgileRL

The complete post-training stack for LLMs, plus RL for any problem. SFT, DPO and multi-turn agentic RL, with evolutionary hyperparameter optimization built in.

hyperledger-cello / cello

Operating System for Enterprise Blockchain

optuna / optuna-examples

Examples for https://github.com/optuna/optuna

whitecircle / halo

Halo is an open-source framework built by White Circle for training large language and multimodal models

fyoorer / ShadowClone

Unleash the power of cloud

internetarchive / brozzler

brozzler - distributed browser-based web crawler

markqvist / LXMF

A universal, distributed and secure messaging protocol for Reticulum

koordinates / kart

Distributed version-control for geospatial and tabular data

lablup / backend.ai

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

redai-studio / Relax

An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale