incoai / splash
A local inference engine for Apple silicon, built around the model.
incoai / splash
VecSzn / PolyStrata
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent
RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。
Luce-Org / lucebox
LLM speculative inference server for heterogeneous hardware & consumer GPUs
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K context on 6x V100-16GB; also runs on 2x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 220 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving, built-in web UI.
JakeATX / llamAmpere
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
tsaipifong / whirl-llm
WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.
poisonxa16 / pxa
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.
headpiece747 / ninfer-5090-windows
Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context
1314521gjy / cyber-on-strata
在 8 GB 显存的显卡上跑 35B MoE（Cyber-Tiel-Coder-35B-A3B）：专家权重流式上卡、未命中专家由 CPU 池就地计算。Strata 引擎的 fork（MIT）。RTX 4080S 实测 8 GB 档 57.7–63.5 tok/s、全常驻 134–136 tok/s。Build from THIS repo — upstream does not support this model.
publicExcess / splish
Splish: a fork of Inco's Splash, tuned for Apple M5 GPUs. Apache-2.0
omgitsbase / llmash
Ollama-compatible LLM server on a tuned llama.cpp fork. Same commands, API and model store as Ollama, up to 4x Ollama's decode speed and 2.8x vLLM's in our benchmarks, from custom CUDA kernels, MTP speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA.