ahtx / kindi
git clone https://github.com/ahtx/kindi.git
A custom LLM inference engine for low-cost NVIDIA GPUs: Quadro K420 and K1200, two K1200s joined into one 8 GB card, GTX 1080 and RTX 3080. Runs Bonsai 2B, 4B, 9B and 27B models with an OpenAI-compatible server.
+6
stars, 24h · #5,551
+34
stars, 7 days · #5,576
+34
stars, 30 days · #17,480
34
★ total · +34% this week
8
forks · 3 open issues
Star history (as recorded here)
LanguagePython
LicenseAGPL-3.0
Created2026-10-20 (3d ago)
Last pushtoday
OwnerUser
GitHub topics
amperebitnetcudakeplerllm-inferencelocal-llmlow-cost-gpumaxwellopenai-compatiblepascalquadrotensor-parallelismternary
Links
- GitHub: github.com/ahtx/kindi
- Homepage: https://amirhusain.com/kindi/
- Releases · Issues · Hacker News mentions