jpezzulli / sglang-rtxpro6000
git clone https://github.com/jpezzulli/sglang-rtxpro6000.git
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
+0
stars, 24h · #30,788
+~10
stars, 7 days · #12,286
+~45
stars, 30 days · #13,993
175
★ total · +6% this week
26
forks · 6 open issues
Star history (as recorded here)
LanguagePython
LicenseApache-2.0
Created2026-08-25 (2mo ago)
Last pushtoday
OwnerUser
GitHub topics
blackwelldflash2fp8fr-spechicachellm-inferencelocal-llmlong-contextnixlnvfp4qwenqwen3qwen3-8qwen3-8-27bqwen3-8-flash-nextqwen38rtx-pro-6000sglangsm120speculative-decoding
Links
- GitHub: github.com/jpezzulli/sglang-rtxpro6000
- Homepage: https://msoexpert.com
- Releases · Issues · Hacker News mentions
README
Loading README…