openvinotoolkit / openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

sauravsingla / MemVanta

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.