lyogavin / airllm

Python library that runs large language model inference layer by layer, so 70B-class models fit on a single 4GB GPU without quantization.

KittenML / KittenTTS

Open-source text-to-speech library with a 1.7B 1-bit speech model and lightweight ONNX models that run on CPU.

trymirai / uzu

Inference engine for running AI models locally inside apps, with Rust core and Swift, Python, and TypeScript bindings.

NVIDIA / Model-Optimizer

A library of model optimization techniques (quantization, pruning, distillation, NAS, speculative decoding, sparsity) that exports compressed checkpoints for inference frameworks.

FluidInference / FluidAudio

Swift SDK running open-source speech models (ASR, TTS, VAD, speaker diarization) locally on Apple devices via CoreML and the Neural Engine.

tensorflow / tensorflow

An end-to-end open source platform for building and deploying machine learning models, with Python and C++ APIs.