52Hz-kiko / mini-vllm-qwen3-serving
A Mini-vLLM optimization project for Qwen3-32B TP=2 serving, Prefix Cache, and Triton PagedAttention.
52Hz-kiko / mini-vllm-qwen3-serving
poorpaper / light-vllm
A small, composable LLM inference runtime built around explicit contracts.
openJiuwen-ai / agent-infer
vLLM based inference framework for agentic workload.
MiaAI-Lab / Qwen3.8-27B-NVFP4-RTX-5090
Qwen3.8-27B-NVFP4 on a Single RTX 5090 (vLLM)
0xSero / DeepSeek-V4.1-Flash-Two-Sparks
DeepSeek-V4.1-Flash on two DGX Sparks: EXL3 routed experts, 262k context, 2M-token KV, vision, tools, MTP, bundled with Pi
jiayuqi7813 / DeepSeek-V4-Flash-0731-CRACK-2x-DGX-Spark
Reproducible refusal-subspace editing and TP=2 deployment of DeepSeek V4 Flash 0731 on two DGX Sparks
kindlingai / glm-5.3-flash-gx10
GLM-5.3-Flash (NVFP4) at TP=4 across 4x ASUS GX10 / GB10 - vLLM + mentat instead of Ray, 524k context, DFlash2. Minimal working recipe.
Eliovp-BV / paiton-vllm-plugin
—
poisonxa16 / pxa
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.
Sentinel3252 / Native-Breeze-TTS-2-Support-for-vLLM-Omni
—
Morrowmake / glm53-flash-cmp170hx-recipe
GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, one-command container install
modelsphere / modelsphere
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.
SlyBase / vllm-sly-radiance
Working on the fastest vllm path for AMD Radeon AI PRO R9700
kikoncuo / jevfire
JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.
ikermoel / open-alternative-jev
Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks
juliankang4 / spark-scope
Read-only dashboard and rack panel for NVIDIA DGX Spark-class nodes, other NVIDIA GPU Linux machines and Apple Silicon Macs running vLLM, SGLang, TensorFold, llama.cpp, Strata or oMLX
ciprianveg / gb10-vllm
vLLM inference solutions for NVIDIA GB10 (DGX Spark, sm121) — KIMI-K3 B12X_MLA + DSpark, GLM-5.2
Enntity / sparkglm
Fast, capable GLM-5.3-Flash FP4-Local on two DGX Sparks: opinionated Atlas serving and reproducible inference research.
BIRENSUPA / vllm-supa
—
Reederey87 / glm53-flash-exl3-2x-dgx-spark
GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode
joeynyc / MiniMax-H3-DGX-Spark
Measured SM121 compatibility recipe for MiniMax H3 FL2VA on one NVIDIA DGX Spark with vLLM-Omni and online FP8.
samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
InferMatrixLab / system1-omni
community maintained vllm/vllm-omni style inference engine for system1 models in Rust for extreme efficient performance
nopnoping / micro-vllm
A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
SlugLab / HBFSim
HBFSim (HBF Sim): an open-source High-Bandwidth Flash (HBF) simulator. Runs real LLM inference on a real GPU and injects modeled HBF timing, capacity and thermal effects while the workload executes.
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.
BIRENSUPA / vllm-omni-supa
—
infercrane / infercrane
Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind one endpoint.
gitcommit90 / glm-5.3-one-spark
GLM-5.3-Flash EXL3 2.05 on one DGX Spark — TP1 vLLM, DFlash2 K5, concurrency, 262K context, and published fidelity measurements
sswen0427 / vllm_notes
—
apimartqfbi49xip / vllm-migration-guide
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
openfabric-systems / simllm
Network-faithful simulation of LLM serving and training deployments
0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark
Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.
zzpanic / qwen3.6-vllm-gfx1201-launchers
—
tonyd2wild / The-Sparky-Command-Center
Plug-and-play, dependency-free web command center for DGX Spark fleets (and any GPU boxes): live per-node GPU/CPU/mem/temps/power, optional RouterOS switch monitoring, and per-model decode/prefill/TTFT/KV cards. Point it at your own hardware.
tonyd2wild / MiMo-V2.6-Flash-DGX-Spark-Recipe
MiMo-V2.6-Flash-RL on 2x NVIDIA DGX Spark (GB10): vLLM TP2 + DFlash + vision/audio, fp8 KV 300K, four fixes for the stock image, measured baselines
apimartymtjlijx / vllm-alternatives
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
MikeCaldera / intel-arc-pro-b70-qwen38-vllm
Reproducible Qwen3.8-27B GPTQ INT4 + MTP4 inference on Intel Arc Pro B70 using vLLM XPU — 84.65 tok/s median.
lukeckprobierts / Hemmingway-1
Hemmingway-1: a 27B model that writes the way a person writes. Open weights, free for non-commercial use (CC BY-NC 4.0); commercial use by agreement.
apimartha / ollama-vs-vllm
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
goehou / enterprise-kb-qa
企业知识库智能问答系统:Qwen2.5 + vLLM + RAG + LangChain Agent + LoRA 微调,一个项目打通大模型应用全链路(部署/RAG/Agent/微调/容器化
axem-solutions / shaide
Self-hosted, distributed, multi-model LLM inference
sudoingX / dgx-spark-deepseek-v41
DeepSeek-V4.1-Flash on two NVIDIA DGX Sparks: EXL3 weights from the official checkpoint, upstream vLLM with our patches and decode kernels
1CatAI / 1Cat-vLLM-Gaudi
1Cat-vLLM on Intel Gaudi
caiovicentino / eikos
Open, calibrated, single-pass typed-decision models (4B & 27B) for finance and trading
seanyourhighness / vllm-sm12x-nvfp4-dflash2
All-NVFP4 vLLM + DFlash2 K7 for Blackwell (SM120/SM121): NVFP4 target/draft/KV, 262K context, optional CPU vision sidecar.
christopherowen / dgx-spark-networking
Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes
lora-sys / nano-vllm-interactive-guide
面向开发者与初学者的 nano-vLLM 交互式源码教程 - 通过 13 个 HTML 互动实验 + 13 章中文教程理解大模型推理引擎
chiro2001 / deepseek-v4.1-flash-ascend910B
DeepSeek-V4.1-Flash W4A8 optimized vLLM deployment for Ascend 910B (A2) / 910C (A3): patches, one-command serve, self-check and acceptance
knapcio / GLM-5.3-Flash-4x-DGX-Spark-TP4
GLM-5.3-Flash NVFP4 on 4x NVIDIA DGX Spark: vLLM TP4, DFlash2 adaptive draft, measured FP8 vs NVFP4 vs EXL3 vs SGLang
bkvargyas / r9700-stack
AMD R9700 Tuned Stack for vLLM and ROCM10
Datura-ai / lium-localmaxxing
Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured cost per million tokens
manjunathshiva / opendecider
Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVIDIA, Apple Silicon, LM Studio, Ollama and vLLM.
hikarioyama / ParallelHue
Truthful scheduler-step and streaming visualization for local LLM inference
feiguangba / minivllm
图解 vLLM 推理引擎：100 课动手练习册 + ~500 行可运行的迷你推理引擎 | An illustrated, hands-on course on how vLLM works
fishensw / VLLM-SM75
一个Turing专用vLLM分支，通过整合FlashInfer 0.6.18（PR #3526）与SM75后端门槛补丁（PR #47949），持续同步vLLM主线最新版本
blockbrain-ai / cygnet-recipe
Typed decisions from frozen Gemma-4-12B-it: one-token option-letter readout over stock vLLM 0.30.0 (JevBench package)
MiaAI-Lab / Ornith-1.5-35B-A3B-DGX-Spark
Serve Ornith-1.5-35B-A3B (NVFP4 + in-checkpoint MTP, 256k ctx) on one DGX Spark — vLLM b12x recipe plus CUDA-graph fixes.
r0b0tlab / DeepSeek-V4-Flash-DSpark-v026-SM121
DeepSeek-V4-Flash-DSpark optimized vLLM 0.26.0 SM121 dual-GB10 evidence (NVFP4 KV, B12X, DSpark K6)