πŸ“ˆ GH Trending Explorer 🎲 Surprise me

Category: llm-tooling Β· clear

lyogavin / airllm

Python library that runs large language model inference layer by layer, so 70B-class models fit on a single 4GB GPU without quantization.

β–² 1.5k/7d β˜… 40k Jupyter Notebook πŸ”₯ today
trymirai / uzu

Inference engine for running AI models locally inside apps, with Rust core and Swift, Python, and TypeScript bindings.

β–² 198/7d β˜… 2.5k Rust πŸ”₯ today
NVIDIA / Model-Optimizer

A library of model optimization techniques (quantization, pruning, distillation, NAS, speculative decoding, sparsity) that exports compressed checkpoints for inference frameworks.

β–² ~79/7d β˜… 5.4k Python πŸ”₯ today
zml / zml

A Zig inference framework that compiles ML models through MLIR and OpenXLA to run on NVIDIA, AMD, Intel, TPU, and Trainium hardware.