ggml-org / llama.cpp
LLM inference in C/C++
ggml-org / llama.cpp
mozilla-ai / llamafile
Distribute and run LLMs with a single file.
RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
lemonade-sdk / lemonade
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
ikawrakow / ik_llama.cpp
llama.cpp fork with additional SOTA quants and improved performance
oceanbase / seekdb
The AI-Native Search Database. Best for agent storage, it unifies vector, text, structured, and semi-structured data into a single engine. This all-in-one database makes agents smarter, easier to run, and more stable.
b4rtaz / distributed-llama
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
Luce-Org / lucebox
LLM speculative inference server for heterogeneous hardware & consumer GPUs
ROCm / FastFlowLM
Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.
UbiquitousLearning / mllm
Fast Multimodal LLM on Mobile Devices
Felix3322 / PotPlayer_ChatGPT_Translate
[support ollama/other model]【兼容ollama和其他模型】将任何具有OpenAI API调用方法的模型集成到PotPlayer中。它使你在观看视频时能够实时翻译字幕，从而打破语言障碍，提升你的观看体验。 This real-time subtitle translation plugin integrates OpenAI's ChatGPT API (or any model with the same API calling method) into PotPlayer. It enables you to translate subtitles on-the-fly while watching videos.
feder-cr / jev
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
go-skynet / go-llama.cpp
LLama.cpp golang bindings
spiritbuun / buun-llama-cpp
Experimental llama.cpp fork for inference research and development
Soulmate-Halo / heterogeneous-gpu-pd-lab
使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
invergent-ai / surogate
Train and serve LLMs at extreme speed and massive throughput.
Helldez / BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
agentty-org / agentty
AI pair programming in your terminal — one static binary, sub-ms startup, any model
cobanov / PS5LM
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
maxbbraun / llama4micro
A "large" language model running on a microcontroller
brontoguana / krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
abhi245y / PS5LM-Plus
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
stew675 / llama-cpp-rdna-boosts
A set of performant RDNA/HIP patches against llama.cpp focusing on high speed inferencing with numerics assurance
JakeATX / llamAmpere
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
poisonxa16 / pxa
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.
nasone32 / llama.cpp-RDNA3-7900xtx-opt
—
QingYis / llama-7900xtx-qwen3.8-27b
—
sixvolts / llama-halo-hybrid
llama.cpp modified to work with Strix Halo + dGPU as an accelerator, Dual Strix Halo Machines, or both
dollarwong / ninfer-v100
NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). Fork of Neroued/ninfer with sm70 vision-encoding speedup.
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): large MoE Qwen3.8-Flash-Next with MTP (prefill 3464 tok/s at 200K context, 6x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 225 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving.
sauravsingla / MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.
miltos22 / llama-wackMall
Caching experts made model agnostic for llama. Drastically speeds up generation speed on MoE models that fully fit in the RAM but partially in Vram. WIP: Same speedup but for models that don't fit in the ram either.
deepgrove-ai / llama.cpp
—
DeepSeekV4Flash0731 / DeepSeek-V4-Flash-0731
DeepSeek V4 Flash 0731 - free deepseek v4 flash download, gguf, ollama, deepseek v4 pro 0813. deepseek v4 flash vs pro, benchmark, MoE, 1M context, compressed sparse attention, coding. CUDA/CPU, no API key, Windows local AI. Download:🡇
milpster / gfx906-llama-cpp
—