CheshireMew / VoxWeave

Local-first high-quality offline RVC voice conversion workstation

dnv2003 / v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

chadhurley25075-png / pd-bridge

Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

Blackwellboy / model-serving-minefield

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

jonathanhawkins / microduck-lab

Train RL policies for the Pollen Microduck 🦆 on an ordinary Mac, no CUDA GPU, and watch them learn live in the browser.

yanght27 / GPU-Perf-Playground

GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构

NoizAI / YuE2-Turbo

Fast, concurrent inference for YuE2. Same model and recipe: 1.68× faster per song, 3.31× more songs per GPU.

sohu-mptc / FlashRec

FlashRec is a CUDA-graph engine for generative recommendation: wide beam search (3–5 SID steps, n=50–512+) over a trie-constrained catalog, in-process FP8 serving, and ranked beams on /v1/chat/completions.

professorpalmer / bonsai-ada-surgery

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

NVIDIA-BioNeMo / BioNeMo-Inference-Runtime

Easy, fast, and memory-efficient structure prediction inference

swax / dub-anything

A tool that allows AI to direct the dubbing process, uses the original actor voices and does the audio generation on your local PC

NguyenSiTrung / VieNeuTTSApp

Cross-platform, fully-offline Vietnamese/English text-to-speech desktop workstation powered by VieNeu-TTS v3 Turbo (ONNX/CUDA, Voice Cloning, EPUB Audiobooks, PySide6/QML)

nandometzger / metal-gauss

Metal-native differentiable 3D Gaussian Splatting for Apple Silicon: fused forward+backward rasterizer on PyTorch MPS, no CUDA anywhere.

ai-infra-bench / ai-infra-bench

Evaluating frontier models on real-world ai-infra engineering tasks

SlugLab / HBFSim

HBFSim (HBF Sim): an open-source High-Bandwidth Flash (HBF) simulator. Runs real LLM inference on a real GPU and injects modeled HBF timing, capacity and thermal effects while the workload executes.

NaiveAI-Labs / NaiveRT

NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.

Quentincls / localmesh-engine

Photographs to a textured mesh on one 8 GB NVIDIA card, offline. Built on TRELLIS.2, with a four view path, and the photographs' own pixels projected onto the texture so printed text stays readable.

alphastorm / omp-ninfer

Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.

arpanpathak / gpu-parallel-book

📖 GPU parallel programming from the ground up : CUDA C++ and Rust 🦀 kernels paired with clickable SVG 2D/3D architecture diagrams (warps, SMs, memory tiers). Includes reduction, scan, matmul, and a full image‑processing capstone

christopherowen / dgx-spark-networking

Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes

ECLIPSE-AI4Science / gradsolve

Differentiable ensemble solvers for differential equations on GPUs, in JAX

zhang677 / PTXBench

PTXBench: an open-source environment for kernel agents with architecture-specifc CUDA/PTX

nibor1896 / Crow

Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen-Image. Runs on crow-nest or llama.cpp on one GPU, or any model via OpenRouter. Windows and Linux.

bertaye / agentic-cuda-optimizer

A langgraph based workflow with a C++ CUDA harness to optimize CUDA kernels

hbofz / NeSLE

GPU-native NES emulator + RL stack: thousands of parallel NES envs on one CUDA GPU, training agents on Super Mario Bros.

ExVRAM / exvram-lab

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

kadenball / qwen38-27b-rtx3060-dcfr

RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.

SoundsGoodAI / fast-gpu-asr

High-throughput batched ASR for NVIDIA GPUs. Run Zipformer and NVIDIA NeMo Parakeet models through one Python API with TensorRT, custom CUDA plugins, GPU beam search, and word timestamps - up to 25,000 RTFx on B300.

MiaAI-Lab / Ornith-1.5-35B-A3B-DGX-Spark

Serve Ornith-1.5-35B-A3B (NVFP4 + in-checkpoint MTP, 256k ctx) on one DGX Spark — vLLM b12x recipe plus CUDA-graph fixes.

c8dhjp4tyv-bit / hawkpoint-npu-llm

Run LLMs and Ollama Qwen across AMD XDNA1 NPU, CPU, and GPU on Ryzen AI Linux systems.

tie-pilot-qxw / pdl-megakernel-reconstruction

Code and reproducibility artifacts for reconstructing megakernel persistence with CUDA PDL

collabray / wisense

WiFi CSI human sensing for Python presence, fall detection, breathing rate, activity classification, and occupancy counting from an ESP32. No camera, no cloud, no PyTorch/CUDA at runtime.