AlexsJones / llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

midagedev / bloomery

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

truespar / paddock

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

Dyluhn / R9V

Shape-specialized ROCm inference for Qwen3.8 Flash Next on RDNA4

nobodywho-ooo / nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

lateos-ai / reflex

A high-performance, GGUF-native Rust & CUDA inference engine optimized for cold-start latency and real-time 'System 1' agent decision loops.

antonellof / frink

Pure-Rust GGUF inference engine with quantized CPU, Metal & CUDA kernels, MoE support, OpenAI-compatible server. Benchmarked head-to-head against llama.cpp.

Michael-A-Kuykendall / shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.