sauravsingla / MemVanta

git clone https://github.com/sauravsingla/MemVanta.git

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.