kvcache-ai / Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

jmaczan / tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

poisonxa16 / pxa

The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.

omgitsbase / llmash

Ollama-compatible LLM server on a tuned llama.cpp fork. Same commands, API and model store as Ollama, up to 4x Ollama's decode speed and 2.8x vLLM's in our benchmarks, from custom CUDA kernels, MTP speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA.