alphastorm / omp-ninfer
git clone https://github.com/alphastorm/omp-ninfer.git
Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.
+0
stars, 24h · #24,864
+~0
stars, 7 days · #25,339
+~0
stars, 30 days · #31,692
45
★ total
4
forks · 3 open issues
Star history (as recorded here)
LanguagePython
LicenseMIT
Created2026-08-27 (1mo ago)
Last push4d ago
OwnerUser
GitHub topics
ai-agentcoding-agentcoding-assistantcudadirectstoragelocal-ailocal-llmnvidiaoh-my-piopenai-compatible-apiopenai-responses-apipersistent-kv-cacheqwenqwen3rtx-3090rtx-4090rtx-5090self-hostedspeculative-decodingstateful-inference