NVlabs / Sana
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer
NVlabs / Sana
NVlabs / LongLive
Long Video Gen / World Model / World Action Model (Embodied) Infrastructure
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
intel / auto-round
PTQ-Level Efficiency, QAT-Level Accuracy|่ฎญ็ปๅ้ๅ็ๆ็๏ผ้ๅๆ็ฅ่ฎญ็ป็็ฒพๅบฆ
microsoft / Tutel
Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4
NVIDIA / cudnn-frontend
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
Avarok-Cybersecurity / atlas
Pure Rust Inference Engine
MiaAI-Lab / Qwen3.8-Flash-Next-Dual-DGX-Sparks
Qwen3.8-Flash-Next-NVFP4-vLLM-Dual-DGX-Spark
tonyd2wild / GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark
GLM-5.3-Flash (NVFP4) on 2x NVIDIA DGX Spark - vLLM TP2, 262K context, MTP. World-first deploy recipe: 7 day-0 bugs found and fixed, patched sm121 image, probes and full report.
hasso5703 / dgx-spark-qwen38
One command turns a DGX Spark or ASUS Ascent GX10 into a private AI server: seven measured, pinned AI lanes (text up to 1M tokens, images, video with sound), run from a local cockpit on a laptop or a phone. Nothing leaves the box.
mmastrac / djev-spark
DiffusionGemma NVFP4 structured decisions on a DGX Spark: container recipe
sergqwer / strata-nvfp4
Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.
dnv2003 / v100-skinny
Hand-written NVFP4 W4A16 CUDA kernels for Volta
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
0xSero / deepseek-v4-flash-0731-spark-sparkinfer
Pinned one-DGX-Spark Docker recipe for DeepSeek V4 Flash with EXL3, SparkInfer, and 262K NVFP4 MLA KV cache
truespar / paddock
Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio
tonyd2wild / Qwen3.8-Flash-Next-NVFP4-DGX-Spark
Qwen3.8-Flash-Next (NVFP4) on DGX Spark in vLLM: one Spark 43.9 tok/s with our disk-backed n-gram table patch, staged gather and reduced-vocab MTP draft; TP2 SPEED 53.7; TP4 CONTEXT 9M KV pool. Launchers, patch, 40-prompt harness, KV ledger, credits. SGLang TP2 day-0 lane kept.
alexellis / glm-5.3-flash-4x-dgx-spark-switchless
GLM-5.3-Flash (NVFP4) at TP4 across 4x DGX Spark via a switchless RoCE ring + DFlash2 โ reproducible recipe
MiaAI-Lab / Qwen3.8-27B-NVFP4-RTX-5090
Qwen3.8-27B-NVFP4 on a Single RTX 5090 (vLLM)
MiaAI-Lab / GLM-5.3-Flash-NVFP4-Dual-DGX-Spark
2x DGX Spark recipe for LibertAIDAI/GLM-5.3-Flash-NVFP4 (NVFP4, Ray TP=2, multimodal)
kindlingai / glm-5.3-flash-gx10
GLM-5.3-Flash (NVFP4) at TP=4 across 4x ASUS GX10 / GB10 - vLLM + mentat instead of Ray, 524k context, DFlash2. Minimal working recipe.
MiaAI-Lab / GLM-5.2-NVFP4-AQLM-Triple-DGX-Sparks
GLM-5.2 NVFP4+AQLM on 3ร DGX Spark โ 380k context MTP serve stack
headpiece747 / ninfer-5090-windows
Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context
0xBakeer / TandemLLM
An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.
tonyd2wild / GLM-5.3-Flash-NVFP4-1M-KV-4x-DGX-Spark
GLM-5.3-Flash (320B MoE) at TP4 across four DGX Sparks, same day as the model drop: 36 tok/s, 1.26M-token FP8 KV pool, 262K context, MTP spec decode. First TP4 glm5_next outside B200 hardware. Full patched-image recipe + the GB10 memory study.
0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark
Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness โ including why the quantization advantage collapses to +0.2% by c16.
ormandj / sglang-glm53-flash-sm120
SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.
seanyourhighness / vllm-sm12x-nvfp4-dflash2
All-NVFP4 vLLM + DFlash2 K7 for Blackwell (SM120/SM121): NVFP4 target/draft/KV, 262K context, optional CPU vision sidecar.
knapcio / GLM-5.3-Flash-4x-DGX-Spark-TP4
GLM-5.3-Flash NVFP4 on 4x NVIDIA DGX Spark: vLLM TP4, DFlash2 adaptive draft, measured FP8 vs NVFP4 vs EXL3 vs SGLang
satellitedown / fast-long-context-cinference
One-menu RTX 5090 setup for non-Swift Huihui Qwen3.8-27B NVFP4 with Cinference, MTP-10 and K8V4.
MiaAI-Lab / Ornith-1.5-35B-A3B-DGX-Spark
Serve Ornith-1.5-35B-A3B (NVFP4 + in-checkpoint MTP, 256k ctx) on one DGX Spark โ vLLM b12x recipe plus CUDA-graph fixes.
dg1kjd / sglang-sxm2
Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, MiniMax-H3.
r0b0tlab / DeepSeek-V4-Flash-DSpark-v026-SM121
DeepSeek-V4-Flash-DSpark optimized vLLM 0.26.0 SM121 dual-GB10 evidence (NVFP4 KV, B12X, DSpark K6)
tonyd2wild / DeepSeek-v4-Flash-0731-Vision-DSpark-1M-NVFP4-KV-2x-DGX-Spark
Give DeepSeek V4 Flash eyes. OpenAI-compatible shim + tiny local VLM. Point any harness at one URL and your text-only model accepts images. Zero dependencies.
Weschera / Qwen3.8-27B-NVFP4-DFlash2-DGX-Spark
Reproducible Qwen3.8-27B NVFP4 + DFlash2 recipe and qualification evidence for one NVIDIA DGX Spark
gabrielolympie / sglang-flashnext-sm120
Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis
othexmr / GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE
GLM-5.3-Flash NVFP4 on two or four DGX Sparks cabled as a ring, with no switch: the RiNGSiDE serving recipe
drowzeys / keys-vLLm.0.27.1-GLM-5.3-Flash-NVFP4-NVFP4KV-1M-Context-Abliterated
GLM-5.3-Flash (Z.ai) NVFP4 (LibertAI) + NVFP4 KV, 1M context on 2x DGX Spark GB10 โ vLLM TP2 recipe, walls ledger, benchmarks, needle to 1.04M; abliteration with blackforest_AI next
Hitheshkaranth / Ornith-1.5_A3B_Model_DGX_Spark_Setup
Ornith-1.5-35B-A3B (NVFP4 + FP8 + MTP) on NVIDIA DGX Spark (GB10)
WestWaters / pollard-weights
A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.
gpillon / ignis
OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at ~100 tok/s, built for local coding agents. Rust + CUDA, Windows and Linux.
nguyenthimy2022kg-alt / Qwen-Flash-SM80-170HX
ๅ SM80 / CMP 170HX ็ Qwen3.8-Flash-Next ็คพๅบ vLLM๏ผNVIDIA NVFP4ใGDSใTEP2ใMTP6๏ผๆฏๆ 1๏ฝ10 ๅนถๅ๏ผ่ฏ่กจไธ PCIe ้ไฟกไผๅใไธญ่ฑๆ้จ็ฝฒไธๅฏๆๅปบๆบ็ ใ
r0b0tlab / qwen38-27b-nvfp4-sm121-vllm
Qwen3.8-27B NVFP4 (W4A16 shipped recipe) + MTP on NVIDIA DGX Spark GB10/SM121 โ vLLM v0.27.2rc0, FP8 KV, 262K context, full reproducibility pack
dolf3131 / dgx-spark-llm-serving
Serving large LLMs on a single DGX Spark (121 GiB): Qwen3.8-Flash-Next on TensorFold/SparkFold EXL3 and vLLM NVFP4, Nemotron 3.5 Super VL on vLLM; what ran, what it measured, what did not work
darkdatter / gb10-repo
Qwen3.8-27B on DGX Spark (GB10): SGLang + NVFP4 + DFlash2. Measured recipe, benchmark harness, and the traps that cost real time.