cobanov / PS5LM
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
cobanov / PS5LM
Soulmate-Halo / heterogeneous-gpu-pd-lab
使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
heterodoxin / AbliteratedOutTheBox
llama.cpp fork that abliterates at inference
Luce-Org / lucebox
LLM speculative inference server for heterogeneous hardware & consumer GPUs
Helldez / BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
stew675 / llama-cpp-rdna-boosts
A set of performant RDNA/HIP patches against llama.cpp focusing on high speed inferencing with numerics assurance
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K context on 6x V100-16GB; also runs on 2x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 220 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving, built-in web UI.
hardenedlinux / SonicBoom
A compiler-first PyTorch alternative. MLIR-based Boom IR, Guile scripting, CUDA and CPU. On consumer GPUs it already beats llama.cpp’s LLM decode with the same kernels; the compiler makes the difference. GPLv3.
spiritbuun / buun-llama-cpp
Experimental llama.cpp fork for inference research and development
JakeATX / llamAmpere
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
poisonxa16 / pxa
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.
brontoguana / krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
sauravsingla / MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.
milpster / gfx906-llama-cpp
—
feder-cr / jev
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
omgitsbase / llmash
Ollama-compatible LLM server on a tuned llama.cpp fork. Same commands, API and model store as Ollama, up to 4x Ollama's decode speed and 2.8x vLLM's in our benchmarks, from custom CUDA kernels, MTP speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA.