utilityai / llama-cpp-rs

—

llmmanorg / llmman

Run any agent on any model, models stored as OCI images

lucasjinreal / Crane

A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..

hardrave / NIGHTRUN

Boot your PC straight into an LLM. Rust, UEFI-resident, no operating system underneath.

DingoOz / llm-visuals

Live truecolor terminal dashboard for a locally running LLM: tok/s, TTFT, GPUs, context, MTP acceptance, layer and real expert routing (llama.cpp)

maximpri / mlxtop

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

midagedev / bloomery

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

Vkuparin / gamepause-for-local-ai

GamePause for Local AI: a Windows tray app that pauses LM Studio, Ollama and other local AI servers while you game, and restores them afterward.