lyogavin / airllm

Python library that runs large language model inference layer by layer, so 70B-class models fit on a single 4GB GPU without quantization.

mudler / LocalAI

Open-source local AI engine that runs LLMs, vision, voice, image, and video models behind OpenAI-compatible APIs on CPU or GPU.

ashhart / TensorFold

Serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs using Metal and CUDA kernels.

microsoft / BitNet

Official C++ inference framework for 1-bit (1.58-bit) LLMs, providing optimized CPU and GPU kernels.