kouhxp / gutsy
Local decision model with calibrated probabilities: send a state and yes/no, choice or score questions, get a probability for every option. 0.8B GGUF on CPU, Jev-style API.
kouhxp / gutsy
julianmb / halofpx
High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.
poisonxa16 / pxa
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.
professorpalmer / bonsai-ada-surgery
Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.
monatis / ggmlc
A multi-framework neural network compiler lowering PyTorch, JAX, Flax, and Keras models to portable, high-performance GGML execution
goosehouse-llc / redde
iPhone, iPad and CarPlay client for the Hermes AI agent or any OpenAI-compatible model. Talk or type, watch it work, approve what it runs. No account, no tracking.
sudiptacmd / homedroid
Turn an old Android phone into a home server: SSH, web hosting, Jellyfin with hardware transcoding, Immich, qBittorrent, Home Assistant and a web dashboard. No root.
jceronch1 / Clonar-voz
Clonación de voz y texto a voz 100% local con Qwen3-TTS y llama.cpp. Funciona en CPU y GPU, 10 idiomas, interfaz web en español.
trademav / ramdeck-core-public
Pool RAM across old PCs, Macs, and Android devices into a unified local AI cluster
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
fulvian / engraft-ngram
ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verified on llama.cpp GGUF (Qwen3.8-Flash-Next).
vedaant00 / opendot
A terminal AI agent you can fully undo - every file and shell action is snapshotted and reversible. Model-agnostic (any LLM), and connects to 1000+ app tools and MCP servers.
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.
tjayze / olympiad-problem-forge
Best Open-Source Math Olympiad Problem Generator 2026
princechhimpachhimpasaab-lang / omega-epsilon-problem-forge
Olympiad Problem Generator 2026: Create Infinite Spanish Math Olympiad-Style Challenges
AnonymousNomad / covert-coder
Covert Coder — a local-first sovereign AI development environment for governed models, workflows, execution, evidence, and verification.
Floral-Overcast / steamdeck-pxe-llm
Bare Steam Deck motherboards as diskless, PXE-booted LLM nodes, plus the GPU clock pin that keeps them fast.
shinbunbun / llama-cpp-p100-patches
llama.cpp performance patches developed and measured on Tesla P100 (GP100, sm_60). Most are not Pascal-specific.
Bai-Cute / big-blue-fish-desktop-pet
蓝色大肥鱼 Windows AI 桌宠，内置 4B 本地大模型，以鲸鱼娘人设结合天气、科技新闻和当前应用，通过文字气泡温柔陪伴。支持一键安装与离电暂停推理。Big Blue Fish desktop pet with local LLM.
Davinchy / macuda
CUDA on an Apple Silicon Mac over Thunderbolt with no NVIDIA driver: a C userspace NVIDIA driver (libtinynv) plus cudart/cublas shims running unmodified llama.cpp and stable-diffusion.cpp on an RTX 5090 eGPU
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): large MoE Qwen3.8-Flash-Next with MTP (prefill 3464 tok/s at 200K context, 6x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 225 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving.
midagedev / bloomery
Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.
RightNow-AI / local-kimi
Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.
tardellirs / gemma3moe-builder
Sparse-upcycle a dense Gemma-3-270m into a Mixture-of-Experts (gemma3moe) and serve it on llama.cpp (CPU). Upcycle, train, inspect routing, GGUF.
sauravsingla / MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.
akivet / Mica
A 4B decision model: calibrated probabilities for yes/no, choice and score questions over long inputs.
jhmeel / bigfoot
An assistant that lives next to your cursor on Windows. It sees your screen, talks with you, draws on things, solves the math you write, and operates your computer. Its AI runs on your PC, so it's private and works offline.
nibor1896 / Crow
Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen-Image. Runs on crow-nest or llama.cpp on one GPU, or any model via OpenRouter. Windows and Linux.
rulith-dev / rulith-inference
Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.
Datura-ai / lium-localmaxxing
Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured cost per million tokens
kadenball / qwen38-27b-rtx3060-dcfr
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.
Veloresearch / velocity-local-services
Velocity Local Services: an app for running LLMs locally on Windows and Linux. Load any GGUF model and chat with web search, tools and schematics, let it use your computer by voice, and serve it to opencode, Hermes or any app through an OpenAI-compatible API and MCP. Context up to 32M tokens. No cloud.
nirbhay41120003 / EchoMind
Local-first voice memory assistant — capture speech, transcribe live, and ask a private RAG chatbot grounded in what you've actually said. No cloud required.
estebanstifli / Meet2Notes
Private, local-first AI meeting assistant for transcription, speaker diarization and meeting notes. Open-source, self-hosted alternative to Granola, Meetily, Fireflies.ai, Fathom and Otter.ai.
aic0d3r / qwen38-strix-halo-harness
Local AI coding agent harness for AMD Strix Halo (Ryzen AI Max+ 395): Qwen3.8 125B on the iGPU, NPU side-jobs, crash recovery, logged benchmarks. Works with any OpenAI-compatible engine; cloud optional.
WestWaters / pollard-weights
A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.
jason-jm / wavesubs
Can't find subtitles? Generate SRT/ASS subtitles from any video with local AI and auto-translate them — no internet, free. macOS & Windows.
EldanRing / winnow-inference
Local typed decisions, chat and vision with Winnow-12B and llama.cpp
julianmb / haloq38flash
qwen3.8-flash-next on strix halo — 91g quant, 56 tok/s, 262k context, receipts included
milpster / gfx906-llama-cpp
—
sjespersen / homelab-agent
Started as a €70 used mini PC running an always-on AI agent (Hermes, n8n, Pi-hole, monitoring, Tailscale, encrypted backups). Now with a second stack next to it: a homelabbed GPU server for local AI (Qwen 27B and Flash-Next, image generation, Ollama, one API). All as code.
Vkuparin / gamepause-for-local-ai
GamePause for Local AI: a Windows tray app that pauses LM Studio, Ollama and other local AI servers while you game, and restores them afterward.