mostlygeek / llama-swap
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
mostlygeek / llama-swap
PrismML-Eng / Bonsai-demo
Demo scripts for running PrismML's Bonsai 2 small LLMs (2B to 27B) locally on Mac, Linux, Windows, or CPU.
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
XHToken / Spark-X2.6
Spark-x2.5 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models
janhq / jan
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
SciSharp / LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
sybil-solutions / local-studio
Control panel for VLLM, Sglang, llama.cpp, exllamav3
AtomicBot-ai / Atomic-Chat
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
llmware-ai / llmware
Unified framework for building enterprise RAG pipelines with small, specialized models
xorbitsai / inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
JohnSnowLabs / spark-nlp
State of the Art Natural Language Processing
twinnydotdev / twinny
Open-source AI coding assistant for VS Code. Coding agent, code completion, chat, edits and reviews with local or hosted models. Your models, your infrastructure.
containers / ramalama
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
floneum / kalosm
Instant, controllable, local pre-trained AI models in Rust
intentee / paddler
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
benman1 / generative_ai_with_langchain
Build production-ready LLM applications and advanced agents using Python, LangChain, and LangGraph. This is the companion repository for the book on generative AI with LangChain.
awaescher / OllamaSharp
The easiest way to use Ollama in .NET
ngxson / wllama
WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
jegly / Box
The most advanced, fully offline client-side AI suite on Android today.
shubham0204 / SmolChat-Android
Running any GGUF SLMs/LLMs locally, on-device in Android
huggingface / llm-ls
LSP server leveraging LLMs for code completion (and more?)
lemonade-sdk / llamacpp-rocm
Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration
hybridgroup / yzma
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
Fast-Editor / Lynkr
Streamline your workflow with Lynkr, a CLI tool that acts as an HTTP proxy for efficient code interactions using Claude Code CLI.
Rose22 / openlumara
handwritten harness for AI, not vibecoded, everything is a module/plugin, tailor made for use with local models, tiny system prompt (around 1k by default), support for exclusive llamacpp features, strong security, strict zero-trust-in-the-ai policy (the code hard limits the ai, no amount of prompt engineering will bypass it)
Nathanw1014 / strix-halo-llamacpp
Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.
Michael-A-Kuykendall / shimmy
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.