altic-dev / FluidVoice

Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows, iOS testflight! Android, Linux waitlist open! A local Wispr Flow alternative. Follow https://x.com/fluidvoiceapp for more

justxor / deepseek-local-guide-ru

🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки

anisayari / sparky

🔒 Your private AI assistant, 100% local on Windows: chat, vision, documents (RAG), web search, images, videos with sound, voice, code projects, MCP tools — one-line install, models picked for your PC.

1nikitas / NIKA_desktop

Native AI assistant for macOS, built with SwiftUI and llama.cpp. Run local LLMs offline, work with PDFs, and create custom AI assistants. Optional NIKA cloud integration.

mozilla-ai / llamafile

Distribute and run LLMs with a single file.

abetlen / llama-cpp-python

Python bindings for llama.cpp

xorbitsai / inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

QwenAudio / SenseVoice

Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.

Osmantic / ODS

ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

ARahim3 / DigUp

Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

AtomicBot-ai / atomic-agent

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

off-grid-ai / OGAM

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

Luce-Org / lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

Mobile-Artificial-Intelligence / maid

Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.

antoinezambelli / forge

A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

ggml-org / Llama-macOS

A cosy home for your LLMs.

Roy3838 / Observer

Why observe computer if computer can observe for you

QwenAudio / Fun-ASR

Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.

thetahealth / mirobody

Turn scattered health data into answers you can trace. Self-hosted AI health data engine for lab reports, wearables and genetics, with a built-in MCP server. Run it with Docker and one LLM key, or keep every model local on llama.cpp.

Woolverine94 / biniou

a self-hosted webui for 30+ generative ai

FuJacob / cotabby

Cotabby is local AI autocomplete for your entire Mac. Open source. On device. Everywhere you type.

StayLameBro / backburner

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

ciddwd / overlay-translator

无需 ROOT 的开源 Android 屏幕实时翻译工具，适合游戏、视觉小说和漫画。支持端侧与云端 OCR、离线 LLM、多种翻译服务和文字朗读（TTS），译文可直接显示在画面上。Open-source no-root Android real-time screen translator for games, visual novels, and manga. Supports on-device and cloud OCR, offline LLMs, multiple translation services, on-screen translations, and text-to-speech (TTS).

mybigday / llama.rn

React Native binding of llama.cpp

kennss / SiliconScope

Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

feder-cr / jev

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.

StartLuxLabs / StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

OleksandrChekhovskyi / hax

A minimalist, terminal-native coding agent written in C.

spiritbuun / buun-llama-cpp

Experimental llama.cpp fork for inference research and development

Soulmate-Halo / heterogeneous-gpu-pd-lab

使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。

utilityai / llama-cpp-rs

—

Helldez / BigMoeOnEdge

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

timmyy123 / LLM-Hub

Local LLM, image&video&music generator, run codex with local models on your phone

cobanov / PS5LM

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

oriveo / oriveo

Open-source, bring-your-own-key (BYOK) multi-model AI chat client for iOS, Android and the web. One LLM client for OpenAI, Anthropic, Google Gemini, OpenRouter, DeepSeek and ten more providers, plus any OpenAI-, Anthropic- or Gemini-compatible endpoint — Ollama, LM Studio, llama.cpp, vLLM. Local-first, self-hostable, no account. AGPL-3.0.

llmmanorg / llmman

Run any agent on any model, models stored as OCI images

spark-arena / sparkrun

sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD

Yzw202011 / OmniSpace

本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，暂时无法提交到仓库）

brontoguana / krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

lucasjinreal / Crane

A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..

sgaofen / humanizer-local-model

Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

Wassimyounes01 / qwen38-uncensored

Local Qwen 3.8 27B uncensored Q4_K_M + harvested SYSTEM pack. Official 3.8 weights, not a 3.6 retitle.

Speedstu / CUDA-for-AMD-Windows

Run CUDA-targeted Windows applications on AMD GPUs with ZLUDA + ROCm/HIP.

hardrave / NIGHTRUN

Boot your PC straight into an LLM. Rust, UEFI-resident, no operating system underneath.

ThinkOffApp / CarWatch

Your car as a chat-room agent: Raspberry Pi 5 + dashcam + local AI. CodeWatch's sibling for the garage.

julianmb / q38rocm

Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

Nathanw1014 / strix-halo-llamacpp

Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.

stew675 / llama-cpp-rdna-boosts

A set of performant RDNA/HIP patches against llama.cpp focusing on high speed inferencing with numerics assurance

JakeATX / llamAmpere

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

SergiioB / intel-arc-pro-b70-inference-cookbook

Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!

sudoingX / local-ai-on-your-gaming-gpu

run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and windows, one line to install

yalun753 / moe-l2

MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.

DingoOz / llm-visuals

Live truecolor terminal dashboard for a locally running LLM: tok/s, TTFT, GPUs, context, MTP acceptance, layer and real expert routing (llama.cpp)

Blackwellboy / model-serving-minefield

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

maximpri / mlxtop

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

moznoazachem / giga-pisar

Гига Писарь: бесплатная локальная русская диктовка для macOS на GigaAM v3. Зажал правый ⌘, сказал, отпустил. Giga Pisar: offline Russian dictation for Mac

htobty / ReadyLLM

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

T-Crypt / berth

Decides who gets your GPU. Chat, image and voice engines lease the card in turn instead of grabbing it, so two of them never load at once and crash each other. One Go binary, nothing else to install.