cactus-compute / needle
Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
cactus-compute / needle
drumih / turbo-fieldfare
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
nobodywho-ooo / nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
cosmo-wander-ai / cosmo-edge
Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.
Sentient-OS-Labs / sentient-os
An on-device LLM understands your entire life, then proactively offers to get your work done through computer use.
zephyr4123 / speakmore
SpeakMore：口述成文，划词即译。开源、免费的语音输入与翻译工具，支持 Mac 与 Windows：语音输入、语音翻译、划词翻译。兼容 Gemini、OpenAI、通义千问、DeepSeek、Kimi、智谱 GLM、MiniMax 及任意 OpenAI 兼容服务，语音输入亦可完全在本机运行。附 Typeless 逆向分析。
google-ai-edge / LiteRT-LM
LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
Muesli-HQ / muesli
Muesli: agent-native local meeting transcription + dictation for macOS (Granola + WisprFlow alternative)
AbrahamPaulJ / nightmare-mobile
On-device ComfyUI style node graph for image/video diffusion models on Android. Local SD1.5, SDXL, Flux, Z-image, Anima, Qwen Image 2.1, Krea 2, MOSS Image, Ideogram 4, inpainting, outpainting, controlnet, LoRA, IPAdapter, LivePortrait and WAN 2.2 5B video on the Qualcomm Hexagon NPU. Fully offline, no server, no account.
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
JiangNanGenius / floe-agent
Open-source on-device AI agent workspace designed for iOS/iPadOS and Android: Linux environments, code execution, local web previews and supported local models. Notes, Office, creative tools and BYOK cloud AI. iPad-first. 开源设备端 AI 智能体与本地运行环境。
sarim2000 / pennywiseai-tracker
PennyWise automatically reads transaction SMS messages and transforms them into organized financial data with on-device AI assistance. No manual entry, no cloud processing, complete privacy.
deskmind-ai / deskmind
Small enough to run on your Mac. Smart enough to ask. An open-source computer use agent for macOS, with local models (MLX) that see, decide and act. 得心，应手。
mizorewww / laya-coreml
Local Laya typed decisions on Apple Core ML and Neural Engine. Validated ports, ~5 ms short decisions on M3 Max, reproducible speed and energy benchmarks.
Helldez / BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
wuyhong715 / rikkahub-agent-pure
RikkaHub Agent · Pure — an Android AI agent that survives long runs. Independent hardening pass on the rikkahub on-device agent: usage ledger & budgets, on-demand tools, expert sub-agents, cold memory, chained workflows. Kotlin/Compose, Shizuku/Termux/MCP. QQ group 1030362371.
AbrahamPaulJ / facefusion-mobile
FaceFusion for Android. Offline face swap running on the phone's NPU, GPU or CPU. Qualcomm Hexagon (v68 to v81), Vulkan fallback, no server and no upload.
qualcomm / GenieX
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
ShadowSafin / AndroLLM
Open-source Android AI using LiteRT-LM with hardware acceleration, cloud providers, memory, and voice.
software-mansion / react-native-executorch
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
tetherto / qvac
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
carloslfu / slotstream
Run a 105 GB AI model on a Mac that can't hold it. Slotstream streams Qwen3.8-Flash-Next (125B mixture of experts) from your SSD and caches the busiest experts in memory, so it runs on Macs with 16 to 64 GB. One native Swift binary on MLX and Metal, no Python, offline. Works with Claude Code, Codex and Ollama or OpenAI clients.
nicedreamzapp / claude-code-local
Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl. Muse-Glimmer 30B (now multimodal — reads images, abliterated), Gemma 4 31B, Qwen 3.5 122B (65 tok/s), DeepSeek V4 Flash (1M ctx). Private, offline, airgap-ready. Built for NDA / legal / healthcare workflows.
kennss / SiliconScope
Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking
SharpAI / SwiftLM
⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
leonickson1 / Swiftlet
Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone.
DenisovAV / flutter_edge_ai
On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.
NightMean / OlliteRT
Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source
lokutor-ai / oido
Oído: open-vocabulary speech recognition that fits in a $5 ESP32-S3. 3.7% LibriSpeech WER, no cloud, no NPU.
FluidInference / FluidUse
Local computer use on Apple silicon: on-device typed decisions with laya and form filling with CUA-S1-FORMS, driven through the macOS Accessibility API
gsamat / amanu
Records and transcribes online meetings. Automatically
yanun0323 / Whallm
DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any M-series MacBook
RelicSync / relic
Relic is the open-source, end-to-end encrypted clipboard manager. Everything you ever copy, indexed by local AI, synced across Windows, macOS, Linux, Android, and iPhone, and searchable forever. Self-hostable.
wpydcr / NanoAvatar
Real-time, high-quality talking avatars, even on a old Android phone: 41 FPS with just 103 ms to the first frame.
john-rocky / CoreML-Models
Core ML model zoo for iOS/macOS — PyTorch models converted to ready-to-use .mlpackage, each with a conversion script and SwiftUI sample app. Sibling repos cover Apple's Core AI framework (iOS/macOS 27) and on-device LLMs.
memovai / mimimodel
MimiModel: Agentic LLM on a $5 chip. 100% on-device. Private by design.
tuff-app / TUFF
The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memory, by streaming MoE experts from the SSD.
lattifai / awesome-EmbeddingGemma
EmbeddingGemma2 resources: local search apps, decision models, inference frameworks, tutorials, and datasets. English / 简体中文.
Greninja9257 / LabLLM
A native macOS lab for teaching tiny language models to think — build the architecture, train the weights, and watch a small LLM emerge from scratch, locally on Apple Silicon with custom data, tokenizers, checkpoints, and MLX acceleration.
nirbhay41120003 / EchoMind
Local-first voice memory assistant — capture speech, transcribe live, and ask a private RAG chatbot grounded in what you've actually said. No cloud required.
1nikitas / NIKA_desktop
Native AI assistant for macOS, built with SwiftUI and llama.cpp. Run local LLMs offline, work with PDFs, and create custom AI assistants. Optional NIKA cloud integration.
yzfly / edgejev
离线可用的本地类型化决策：4 核 CPU 单题 15.6ms。Local & offline Jev / System One inference on CPU — ONNX + INT8, no torch at runtime. 支持 laya / kev / PlayJev
thunderbird / thunderbolt
AI You Control: Choose your models. Own your data. Eliminate vendor lock-in.
TheStageAI / TheWhisper
Optimized Whisper models for streaming and on-device use
christopherkarani / Wax
Shared Single-file memory layer for all your agents, sub mili-second RAG over text, photo and video on Apple Silicon.. No Server. No API. One File. Pure Swift
nanguoyu / minirun-app
Run Kimi K3 (2.8 T parameters,1.56 TB) on iPhone 16 Pro? Weights stream from your SSD through a memory budget you set. Free, local, on Mac and iPhone.
chilly23 / RP2040-and-ESP32-AI
Run a 28.9M-parameter TinyLM on ESP32-S3 with an RP2040 OLED display node for fully local embedded AI inference.
NorbertKlockiewicz / on-device-3d-scanner
3D scanner running fully on-device: 8 photos to a spinnable gaussian-splat scan in ~3s on an iPhone, via Depth Anything 3 + react-native-executorch + TypeGPU
https-deeplearning-ai / sc-on-device-ai-memory
—
kjckangshifu / ML-Sharp-QNN
Port Apple's SHARP (single-image 3D Gaussian Splatting) to on-device Android via Qualcomm QNN HTP — fully offline inference on Snapdragon
codejunkie99 / LocalFlow
Fast, private, on-device dictation for Apple silicon Macs
shyringo / deepseek-v4-flash-0731-in-c
Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, resident OpenAI-compatible API with function tools. | 在笔记本单颗 CPU 上本地运行原生 284B-A13B DeepSeek-V4-Flash-0731 大模型：纯 C，最低 8 GB 内存，无需 GPU，最优 TPOT 0.892 秒/token，支持模型常驻的 OpenAI 兼容接口与函数工具。
rayl15 / Godwit
Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no dependencies.
RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
woheller69 / whisperIME
Android Input Method Editor (IME) based on Whisper
feder-cr / jev
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.