mw00 / project-maya

Project Maya - run GLM-5.3-Flash (321B MoE) and DeepSeek-V4.1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

jamiepine / voicebox

The open-source AI voice studio. Clone, dictate, create.

kanaworksai-hue / audio-studio-mcp

Local AI BGM and sound effects for Claude Code and any MCP client. Stable Audio 3 on Apple Silicon (MLX) and Windows NVIDIA GPUs (PyTorch). No cloud, no API key.

bendlang / bend

Bend 2: a fast language that blocks AI mistakes via proof. Install: curl -fsSL https://bend-lang.com/install.sh | sh

timoncool / YuE2-Studio

Local AI song generator with an editable score — YuE2 on your GPU: full songs with vocals, sheet music, covers, exact replay. Native Windows app, no Python, installer with auto-update.

ace-step / ACE-Step-1.5

The most powerful local music generation model that outperforms almost all commercial alternatives, supporting Mac, AMD, Intel, and CUDA devices.

VecSzn / PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

TrenTorch / TrenTorch

Don't memorize ML. Understand it from first principles including Inference, Cuda, Deep Learning, Data Science and Reinforcement Learning

Indras-Mirror / Xolotl

Xolotl (formerly Strata-DSV4.1): DeepSeek-V4.1-Flash, all 552B, on one gaming PC (RTX 4090 + 90 GB RAM) at ~27 tok/s, plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B. A heavily extended fork of Strata: MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.

alexturkey / voxcpm2-turkce-ses-tasarimi

Turkish voice design with VoxCPM2 on a 4 GB GPU: describe a voice, pick a variant, read Turkish text. Patched audio.cpp, local UI, 7 synthetic voices.

antirez / ds4

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

syv-ai / HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

plotarmordev / drama-tensorfold

Fish Audio Drama 3 text-to-speech on the TensorFold native CUDA engine for DGX Spark (GB10): recipe, engine patches, benchmarks. Built with Fish Audio.

NVIDIA / TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

NVIDIA / skills

Agent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end.

datawhalechina / zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

ForceInjection / AI-fundamentals

AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。

sgl-project / sglang-omni

High Performance Serving Runtime For Interactive Super Intelligence System

flashrt-project / FlashRT

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

surya-koritala / Glyd

Lossless AI compression: 33% less GPU memory, bit for bit.

NVIDIA / accelerated-computing-hub

NVIDIA curated collection of educational resources related to general purpose GPU programming.

formetaohy / Neura

Train and run AI in your game. Cross-platform. No CUDA.

NVIDIA / cuda-rust

NVIDIA's CUDA platform for Rust. Host runtime crates plus Tile (cutile-rs) and SIMT (cuda-oxide) kernel programming models in idiomatic Rust.

lxe / dsx

DeepSeek-V4.1-Flash-1017 (552B MoE) on 2x RTX 3090 + 128 GB RAM: Rust + CUDA engine and 2-bit quant. 30-37 tok/s decode.

SemiAnalysisAI / InferenceX

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

LMCache / LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

3145544645 / gouji

构屎 GOUJI — 纯 C++/CUDA 的三值 mini LLM。2 bit/权重，零 Python；联想记忆工具路由，零训练参数；143 个缺陷与 29 个变异测试记录在案。

CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent

RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。

ahtx / kindi

A custom LLM inference engine for low-cost NVIDIA GPUs: Quadro K420 and K1200, two K1200s joined into one 8 GB card, GTX 1080 and RTX 3080. Runs Bonsai 2B, 4B, 9B and 27B models with an OpenAI-compatible server.

gpustack / gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

noonghunna / club-3090

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.

fermionresearch / phonon

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

zhongkaifu / TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

Code-Amadeus / Amadeus

Real-time multimodal desktop agent evolving toward a persistent AI OS interface (0.17 α).

NaiveAI-Labs / NaiveRT

NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.

NVIDIA / cutlass

CUDA Templates and Python DSLs for High-Performance Linear Algebra

vosen / ZLUDA

CUDA on non-NVIDIA GPUs

NVIDIA / warp

A Python framework for GPU-accelerated simulation, robotics, and machine learning.

MrNeRF / LichtFeld-Studio

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

Luce-Org / lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

hashcat / hashcat

World's fastest and most advanced password recovery utility

pegainfer-project / kern

A model-agnostic GPU runtime for shipping models as verified programs.

xlite-dev / LeetCUDA

An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.

catboost / catboost

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

tracel-ai / burn

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

roflcoopter / viseron

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.

tracel-ai / cubecl

Multi-platform high-performance compute language extension for Rust.

NVlabs / kda

Kernel Design Agents (KDA) is a agent-centric workflow to write high-performance CUDA Kernels.

BBuf / how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

Wallawalla47 / Infernix

C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer

midagedev / bloomery

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing

NVIDIA / cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

shader-slang / slang

Making it easier to work with shaders

h5i-dev / h5i

An Axum-based Rust web framework for building and verifying web apps with Lean 4, featuring optional GPU acceleration and integrated development infrastructure for red-teaming and CI/CD.

NVIDIA / TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

NVIDIA / cuopt

GPU accelerated decision optimization

NoizAI / YuE2-Turbo

Fast, concurrent inference for YuE2. Same model and recipe: 1.68× faster per song, 3.31× more songs per GPU.

NVIDIA-BioNeMo / BioNeMo-Inference-Runtime

Easy, fast, and memory-efficient structure prediction inference