lyogavin / airllm

Python library that runs large language model inference layer by layer, so 70B-class models fit on a single 4GB GPU without quantization.

trymirai / uzu

Inference engine for running AI models locally inside apps, with Rust core and Swift, Python, and TypeScript bindings.

facebookresearch / segment-anything

Inference code, checkpoints, and notebooks for Meta's Segment Anything Model (SAM), which produces object masks from points, boxes, or automatically for whole images.

facebookresearch / audiocraft

PyTorch library for audio generation research, providing inference and training code for MusicGen, AudioGen, and EnCodec.

Effect-TS / effect

TypeScript library for building type-safe production applications with typed errors, dependency injection, structured concurrency, and schema validation.

NVIDIA / Model-Optimizer

A library of model optimization techniques (quantization, pruning, distillation, NAS, speculative decoding, sparsity) that exports compressed checkpoints for inference frameworks.

FluidInference / FluidAudio

Swift SDK running open-source speech models (ASR, TTS, VAD, speaker diarization) locally on Apple devices via CoreML and the Neural Engine.