lyogavin / airllm

Python library that runs large language model inference layer by layer, so 70B-class models fit on a single 4GB GPU without quantization.

VectifyAI / PageIndex

Python library and SDK that builds a hierarchical tree index of documents so an LLM can reason through it for retrieval, without vector databases or chunking.

NVIDIA / Model-Optimizer

A library of model optimization techniques (quantization, pruning, distillation, NAS, speculative decoding, sparsity) that exports compressed checkpoints for inference frameworks.

tensorflow / tensorflow

An end-to-end open source platform for building and deploying machine learning models, with Python and C++ APIs.

pytorch / pytorch

Python library providing GPU-accelerated tensor computation like NumPy, plus a tape-based autograd system for building deep neural networks.