BerriAI / litellm

Python SDK and self-hostable proxy server that calls 100+ LLM providers through a unified OpenAI-compatible API.

ashhart / TensorFold

Serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs using Metal and CUDA kernels.

NVIDIA / Model-Optimizer

A library of model optimization techniques (quantization, pruning, distillation, NAS, speculative decoding, sparsity) that exports compressed checkpoints for inference frameworks.

NVIDIA-NeMo / Nemotron

Developer hub with training recipes, cookbooks, datasets, and end-to-end examples for building with NVIDIA's open Nemotron models.

zml / zml

A Zig inference framework that compiles ML models through MLIR and OpenXLA to run on NVIDIA, AMD, Intel, TPU, and Trainium hardware.