Panchovix / nvidia-vbios-vix-editor

Offline NVIDIA Blackwell VBIOS editor.

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

sgl-project / sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

NVIDIA / TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

NVIDIA / TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

lightseekorg / tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

openlake-project / openlake

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

NVIDIA / dgx-spark-playbooks

Collection of step-by-step playbooks for setting up AI/ML workloads on NVIDIA DGX Spark devices with Blackwell architecture.

NVIDIA / cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

kacper-daftcode / vLLM-Moet

A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards

sergqwer / strata-nvfp4

Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.

jpezzulli / sglang-rtxpro6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

Saganaki22 / ComfyUI-sol-attn

NVIDIA Sol-Attn for ComfyUI / Triton kernel on SM89 - SM121, with zero-copy MiniMax H3 nodes: memory-efficient attention, scheduled tau with graph preview, and feed-forward chunking. Measured 1.14–1.44× vs SageAttention and −37% MLP peak VRAM on H3

Blackwellboy / model-serving-minefield

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

truespar / paddock

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

jesdga95 / basalt

Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell

sybil-solutions / deepseek-v4.1-flash-4x-rtx-pro-6000

DeepSeek V4.1 Flash on 4x RTX PRO 6000 Blackwell: native weights, DSpark, vision, NVMe or locked RAM Engram offload.

KingGore / ComfyUI_sol-attn_Blackwell

Sol-attn comfy plug-in just for Blackwell

samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.

NaiveAI-Labs / NaiveRT

NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.

ormandj / sglang-glm53-flash-sm120

SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.

seanyourhighness / vllm-sm12x-nvfp4-dflash2

All-NVFP4 vLLM + DFlash2 K7 for Blackwell (SM120/SM121): NVFP4 target/draft/KV, 262K context, optional CPU vision sidecar.

shiloz-stack / blackwell-moe-kernel-scheduler

Workload-aware persistent expert-tile scheduling for irregular MoE inference kernels on NVIDIA Blackwell GPUs

ExVRAM / exvram-lab

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

xilla420 / dlss5-wow-sidecar

DLSS 5 Neural Rendering for World of Warcraft (WoW) on NVIDIA RTX 40/50 — a safe overlay with no DLL injection, no hooks and no ReShade next to Wow.exe. Free and open source.

gabrielolympie / sglang-flashnext-sm120

Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis

gpillon / ignis

OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at ~100 tok/s, built for local coding agents. Rust + CUDA, Windows and Linux.

TonyJoaca / DLSSG-Transfusion

Universal DLSS Multi-Frame Generation enabler (2x through 6x and Dynamic Mode) for NVIDIA RTX 40-Series (Ada Lovelace) GPUs with runtime Blackwell Kernel Transfusion.