youssofal / MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

ovg-project / kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

dphnAI / sonar

Large-scale LLM inference engine

datawhalechina / zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

qualcomm / ai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

insight-platform / Savant

Python Computer Vision & Video Analytics Framework With Batteries Included

ARahim3 / mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

pylint-dev / astroid

A common base representation of python source code for pylint and other projects

zengxiao-he / tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

SearchSavior / OpenArc

Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.

InferMatrixLab / vllm-rlt

vllm based inference engine for looped transformer

dnv2003 / v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

peonist-ai / halogen-server

The fastest way to run Qwen3.8 27B on Strix Halo (gfx1151)

0xBakeer / TandemLLM

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.

Generated-Computers / Locals

Run many AI agents at once on your Mac, about three times faster.

InferMatrixLab / vllm-omni-mlx

High-performance OpenAI and Anthropic compatible omni-modality model inference server for Apple Silicon.