tracel-ai / burn

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

smg-project / smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

ai-dynamo / dynamo

A Datacenter Scale Distributed Inference Serving Framework

truespar / paddock

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

safetensors / safetensors

Simple, safe way to store and distribute tensors

sonos / tract

Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference

tensorlakeai / tensorlake

Tensorlake is a serverless runtime for sandboxes and deploying background agentic applications

earth-mover / icechunk

Open-source, cloud-native transactional tensor storage engine