Gxj230958 / TritonAscendBench
A migration benchmark that measures whether production GPU Triton kernels — extracted from vLLM and SGLang inference paths — can be ported to the Huawei Ascend 910B (Atlas A2) via `triton-ascend` without losing correctness or performance.
chadhurley25075-png / pd-bridge
Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
tonyd2wild / DeepSeek-V4.1-Flash-vLLM-DGX-Spark
DeepSeek-V4.1-Flash (552B MoE, MXFP4 experts, 1M ctx) on four NVIDIA DGX Sparks with vLLM TP4: Engram-on-disk patch, sm121 kernel build, launchers, measured numbers
yanght27 / GPU-Perf-Playground
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
tonyd2wild / Qwen3.8-Flash-Next-NVFP4-DGX-Spark
Qwen3.8-Flash-Next (NVFP4) on DGX Spark in vLLM: one Spark 43.9 tok/s with our disk-backed n-gram table patch, staged gather and reduced-vocab MTP draft; TP2 SPEED 53.7; TP4 CONTEXT 9M KV pool. Launchers, patch, 40-prompt harness, KV ledger, credits. SGLang TP2 day-0 lane kept.
magiccodingman / vllm-radiance
Codeberg fork of StillDeadcode / vllm-radiance which is a custom vllm tuned for R9700 GPU's
DominikBucko / qwen38-flash-next-2x3090
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
allover326 / deepseek-v4-cmp170hx
DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallelism, which vLLM does not support upstream.
ruashots / open-h3-ir
Open-source local implementation of MiniMax H3 Context-IR. Turn simple prompts into structured, validated H3 video briefs.
mmastrac / djev
Jev-style structured decisions on DiffusionGemma: the example server from vLLM PR 57250
jayleaton / glm53-tensorfold-spark
GLM-5.3-Flash (abliterated EXL3) on 2x NVIDIA DGX Spark with the TensorFold engine: 1.8x faster decode than vLLM, 4x256k concurrent threads, byte-exact speculative decoding. Work in progress.
NoizAI / YuE2-Turbo
Fast, concurrent inference for YuE2. Same model and recipe: 1.68× faster per song, 3.31× more songs per GPU.
htobty / ReadyLLM
Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.
52Hz-kiko / mini-vllm-qwen3-serving
A Mini-vLLM optimization project for Qwen3-32B TP=2 serving, Prefix Cache, and Triton PagedAttention.
poorpaper / light-vllm
A small, composable LLM inference runtime built around explicit contracts.
openJiuwen-ai / agent-infer
vLLM based inference framework for agentic workload.
Sentinel3252 / Native-Breeze-TTS-2-Support-for-vLLM-Omni
—
jiayuqi7813 / DeepSeek-V4-Flash-0731-CRACK-2x-DGX-Spark
Reproducible refusal-subspace editing and TP=2 deployment of DeepSeek V4 Flash 0731 on two DGX Sparks
kindlingai / glm-5.3-flash-gx10
GLM-5.3-Flash (NVFP4) at TP=4 across 4x ASUS GX10 / GB10 - vLLM + mentat instead of Ray, 524k context, DFlash2. Minimal working recipe.
Eliovp-BV / paiton-vllm-plugin
—
Morrowmake / glm53-flash-cmp170hx-recipe
GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, one-command container install
SlyBase / vllm-sly-radiance
Working on the fastest vllm path for AMD Radeon AI PRO R9700
ikermoel / open-alternative-jev
Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks
ciprianveg / gb10-vllm
vLLM inference solutions for NVIDIA GB10 (DGX Spark, sm121) — KIMI-K3 B12X_MLA + DSpark, GLM-5.2
Enntity / sparkglm
Fast, capable GLM-5.3-Flash FP4-Local on two DGX Sparks: opinionated Atlas serving and reproducible inference research.
Reederey87 / glm53-flash-exl3-2x-dgx-spark
GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode
joeynyc / MiniMax-H3-DGX-Spark
Measured SM121 compatibility recipe for MiniMax H3 FL2VA on one NVIDIA DGX Spark with vLLM-Omni and online FP8.
samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
InferMatrixLab / system1-omni
community maintained vllm/vllm-omni style inference engine for system1 models in Rust for extreme efficient performance
nopnoping / micro-vllm
A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
SlugLab / HBFSim
HBFSim (HBF Sim): an open-source High-Bandwidth Flash (HBF) simulator. Runs real LLM inference on a real GPU and injects modeled HBF timing, capacity and thermal effects while the workload executes.
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.
BIRENSUPA / vllm-omni-supa
—
gitcommit90 / glm-5.3-one-spark
GLM-5.3-Flash EXL3 2.05 on one DGX Spark — TP1 vLLM, DFlash2 K5, concurrency, 262K context, and published fidelity measurements
apimartqfbi49xip / vllm-migration-guide
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
openfabric-systems / simllm
Network-faithful simulation of LLM serving and training deployments
0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark
Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.
zzpanic / qwen3.6-vllm-gfx1201-launchers
—
tonyd2wild / MiMo-V2.6-Flash-DGX-Spark-Recipe
MiMo-V2.6-Flash-RL on 2x NVIDIA DGX Spark (GB10): vLLM TP2 + DFlash + vision/audio, fp8 KV 300K, four fixes for the stock image, measured baselines
apimartymtjlijx / vllm-alternatives
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
MikeCaldera / intel-arc-pro-b70-qwen38-vllm
Reproducible Qwen3.8-27B GPTQ INT4 + MTP4 inference on Intel Arc Pro B70 using vLLM XPU — 84.65 tok/s median.
apimartha / ollama-vs-vllm
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
goehou / enterprise-kb-qa
企业知识库智能问答系统:Qwen2.5 + vLLM + RAG + LangChain Agent + LoRA 微调,一个项目打通大模型应用全链路(部署/RAG/Agent/微调/容器化
RodriMora / dgx-spark-grafana-dashboard
Grafana dashboard for NVIDIA DGX Spark clusters running vLLM: throughput, contention, and GB10 GPU hardware telemetry (NVIDIA-themed, 55 panels)
sudoingX / dgx-spark-deepseek-v41
DeepSeek-V4.1-Flash on two NVIDIA DGX Sparks: EXL3 weights from the official checkpoint, upstream vLLM with our patches and decode kernels
intel / gpu-ai-skills
Agent Skills for running, benchmarking, and profiling Hugging Face models on Intel GPUs — PyTorch, vLLM-XPU, and SGLang-XPU
caiovicentino / eikos
Open, calibrated, single-pass typed-decision models (4B & 27B) for finance and trading
seanyourhighness / vllm-sm12x-nvfp4-dflash2
All-NVFP4 vLLM + DFlash2 K7 for Blackwell (SM120/SM121): NVFP4 target/draft/KV, 262K context, optional CPU vision sidecar.
1CatAI / 1Cat-vLLM-Gaudi
1Cat-vLLM on Intel Gaudi
christopherowen / dgx-spark-networking
Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes
knapcio / GLM-5.3-Flash-4x-DGX-Spark-TP4
GLM-5.3-Flash NVFP4 on 4x NVIDIA DGX Spark: vLLM TP4, DFlash2 adaptive draft, measured FP8 vs NVFP4 vs EXL3 vs SGLang
bkvargyas / r9700-stack
AMD R9700 Tuned Stack for vLLM and ROCM10
chiro2001 / deepseek-v4.1-flash-ascend910B
DeepSeek-V4.1-Flash W4A8 optimized vLLM deployment for Ascend 910B (A2) / 910C (A3): patches, one-command serve, self-check and acceptance
manjunathshiva / opendecider
Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVIDIA, Apple Silicon, LM Studio, Ollama and vLLM.
xp2001-coder / zero2vllm
llm-inference
hikarioyama / ParallelHue
Truthful scheduler-step and streaming visualization for local LLM inference
blockbrain-ai / cygnet-recipe
Typed decisions from frozen Gemma-4-12B-it: one-token option-letter readout over stock vLLM 0.30.0 (JevBench package)
fishensw / VLLM-SM75
一个Turing专用vLLM分支，通过整合FlashInfer 0.6.18（PR #3526）与SM75后端门槛补丁（PR #47949），持续同步vLLM主线最新版本