anisayari / sparky
🔒 Your private AI assistant, 100% local on Windows: chat, vision, documents (RAG), web search, images, videos with sound, voice, code projects, MCP tools — one-line install, models picked for your PC.
anisayari / sparky
abetlen / llama-cpp-python
Python bindings for llama.cpp
xorbitsai / inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Osmantic / ODS
ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.
antoinezambelli / forge
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows
thetahealth / mirobody
Turn scattered health data into answers you can trace. Self-hosted AI health data engine for lab reports, wearables and genetics, with a built-in MCP server. Run it with Docker and one LLM key, or keep every model local on llama.cpp.
Woolverine94 / biniou
a self-hosted webui for 30+ generative ai
StayLameBro / backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
StartLuxLabs / StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
spark-arena / sparkrun
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD
Yzw202011 / OmniSpace
本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，暂时无法提交到仓库）
sgaofen / humanizer-local-model
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
Speedstu / CUDA-for-AMD-Windows
Run CUDA-targeted Windows applications on AMD GPUs with ZLUDA + ROCm/HIP.
ThinkOffApp / CarWatch
Your car as a chat-room agent: Raspberry Pi 5 + dashcam + local AI. CodeWatch's sibling for the garage.
julianmb / q38rocm
Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.
Nathanw1014 / strix-halo-llamacpp
Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.
SergiioB / intel-arc-pro-b70-inference-cookbook
Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!
sudoingX / local-ai-on-your-gaming-gpu
run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and windows, one line to install
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
htobty / ReadyLLM
Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.
julianmb / halofpx
High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.
professorpalmer / bonsai-ada-surgery
Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.
monatis / ggmlc
A multi-framework neural network compiler lowering PyTorch, JAX, Flax, and Keras models to portable, high-performance GGML execution
jceronch1 / Clonar-voz
Clonación de voz y texto a voz 100% local con Qwen3-TTS y llama.cpp. Funciona en CPU y GPU, 10 idiomas, interfaz web en español.
trademav / ramdeck-core-public
Pool RAM across old PCs, Macs, and Android devices into a unified local AI cluster
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
fulvian / engraft-ngram
ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verified on llama.cpp GGUF (Qwen3.8-Flash-Next).
vedaant00 / opendot
A terminal AI agent you can fully undo - every file and shell action is snapshotted and reversible. Model-agnostic (any LLM), and connects to 1000+ app tools and MCP servers.
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.
RightNow-AI / local-kimi
Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.
tardellirs / gemma3moe-builder
Sparse-upcycle a dense Gemma-3-270m into a Mixture-of-Experts (gemma3moe) and serve it on llama.cpp (CPU). Upcycle, train, inspect routing, GGUF.
akivet / Mica
A 4B decision model: calibrated probabilities for yes/no, choice and score questions over long inputs.
nibor1896 / Crow
Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen-Image. Runs on crow-nest or llama.cpp on one GPU, or any model via OpenRouter. Windows and Linux.
rulith-dev / rulith-inference
Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.
kadenball / qwen38-27b-rtx3060-dcfr
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.
nirbhay41120003 / EchoMind
Local-first voice memory assistant — capture speech, transcribe live, and ask a private RAG chatbot grounded in what you've actually said. No cloud required.
estebanstifli / Meet2Notes
Private, local-first AI meeting assistant for transcription, speaker diarization and meeting notes. Open-source, self-hosted alternative to Granola, Meetily, Fireflies.ai, Fathom and Otter.ai.
WestWaters / pollard-weights
A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.
EldanRing / winnow-inference
Local typed decisions, chat and vision with Winnow-12B and llama.cpp
sjespersen / homelab-agent
Started as a €70 used mini PC running an always-on AI agent (Hermes, n8n, Pi-hole, monitoring, Tailscale, encrypted backups). Now with a second stack next to it: a homelabbed GPU server for local AI (Qwen 27B and Flash-Next, image generation, Ollama, one API). All as code.