ollama / ollama

Runs open-weight LLMs such as Gemma, Qwen, DeepSeek, and gpt-oss locally or in the cloud, with a CLI, REST API, and Python and JavaScript libraries.

vllm-project / vllm

A high-throughput LLM inference and serving engine featuring PagedAttention, continuous batching, and an OpenAI-compatible API server.

sgl-project / sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

StarTrail-org / LEANN

[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.

lightseekorg / tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

Quincunx33 / Ai-jailbreak

A collection of jailbreak prompts and exploit techniques for local and frontier AI models, with modern methods for Qwen3.5, Gemma 4, Llama 4, Kimi K3, GPT-OSS, GPT-5.x, Gemini 3.x and Grok 4.x. For red-teaming and AI safety research only.

openai / gpt-oss

gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

OpenPipe / ART

Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement learning for Qwen3.6, GPT-OSS, Llama, and more!

papersgpt / papersgpt-for-zotero

A powerful Zotero AI and MCP plugin with ChatGPT, Gemini 4, Claude Fable 5.1, Claude Opus 5.5, DeepSeek V4.1, Grok, OpenRouter, Kimi k3, GLM 5.3, SiliconFlow, GPT-oss, Gemma 4, Qwen 3.8

AtomicBot-ai / Atomic-Chat

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

NVIDIA-NeMo / Automodel

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

oumi-ai / oumi

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

qualcomm / GenieX

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

InternLM / xtuner

A Next-Generation Training Engine Built for Ultra-Large MoE Models

johnbean393 / Sidekick

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

curiousily / AI-Bootcamp

Self-paced bootcamp on Generative AI. Tutorials on ML fundamentals, Ollama, LLMs, RAGs, LangChain, LangGraph, Fine-tuning, DSPy & AI Agents (CrewAI), (Using ChatGPT, gpt-oss, Claude, Qwen, Gemma, Llama, Gemini)

rexmhall09 / TUFF

The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memory, by streaming MoE experts from the SSD.

aurafhe-official / mcp

Connect AI assistants to Aura encrypted numeric computation. Explore Aura AI, its published GPT-OSS-20B benchmark and public learning demos.

rayl15 / Godwit

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no dependencies.