MemTensor / MemT-1.0
Official repository for MemT-1.0
MemTensor / MemT-1.0
eagle-hq / EAGLE-4
Official Inference Code of EAGLE-4
sgl-project / sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
xorbitsai / inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
OpenDCAI / DataFlow
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
gpustack / gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
noonghunna / club-3090
Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.
ovg-project / kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
SemiAnalysisAI / InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
intel / auto-round
PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度
sgl-project / sglang-omni
SGLang-Omni is a high-performance serving runtime for interactive intelligence model systems.
datawhalechina / zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
ModelCloud / GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
sgl-project / SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
OpenMOSS / MOVA
A foundation model that generates synchronized video and audio in a single model
Tencent-Hunyuan / UniRL
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning
XYZ-AI-Lab / axrl
AxisRL is an agentic RL post-training framework built on SGLang rollout, Megatron training, and real-world agent workflows.
spark-arena / sparkrun
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD
hasso5703 / dgx-spark-qwen38
One command turns a DGX Spark or ASUS Ascent GX10 into a private AI server: seven measured, pinned AI lanes (text up to 1M tokens, images, video with sound), run from a local cockpit on a laptop or a phone. Nothing leaves the box.
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
Gxj230958 / TritonAscendBench
A migration benchmark that measures whether production GPU Triton kernels — extracted from vLLM and SGLang inference paths — can be ported to the Huawei Ascend 910B (Atlas A2) via `triton-ascend` without losing correctness or performance.
rednote-machine-learning / Inference-autopilot
Evidence-driven SGLang inference deployment optimization.
tonyd2wild / Qwen3.8-Flash-Next-NVFP4-DGX-Spark
Qwen3.8-Flash-Next (NVFP4) on DGX Spark in vLLM: one Spark 43.9 tok/s with our disk-backed n-gram table patch, staged gather and reduced-vocab MTP draft; TP2 SPEED 53.7; TP4 CONTEXT 9M KV pool. Launchers, patch, 40-prompt harness, KV ledger, credits. SGLang TP2 day-0 lane kept.
yanght27 / GPU-Perf-Playground
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
RH-RunningHub / MiniMax-H3-MultiGPU-Lightning
MiniMax-H3 multi-GPU inference acceleration: ~12x on 8x RTX 6000D (step distillation + SageAttention2 + Cache-DiT + torch.compile, TP2+Ulysses4 on sglang)
htobty / ReadyLLM
Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.
sybil-solutions / qwen-serve
Fastest validated sglang recipe for Qwen3.8-27B (AWQ-INT4 + DSpark) on RTX 3090s — 1×/2×/4× GPU presets, image+video, full benchmarks
knapcio / DeepSeek-V4.1-Flash-4x-DGX-Spark-TP4
Tuned TP4 serving profile for DeepSeek-V4.1-Flash on 4x NVIDIA DGX Spark (SGLang, DSpark), built on the MiaAI-Lab recipe
MiaAI-Lab / Qwen3.8-27B-RTX-6000-PRO-SGLang-DSpark
Qwen3.8-27B on SGLang for NVIDIA RTX PRO 6000 (96GB, SM120): DSpark speculative decoding, full 256K context, 8 concurrent requests
baidu-baige / sglang-kunlun
SGLang Kunlun (sglang-kunlun) is a community-maintained hardware plugin designed to seamlessly run SGLang on the Kunlun XPU.
openfabric-systems / simllm
Network-faithful simulation of LLM serving and training deployments
ormandj / sglang-glm53-flash-sm120
SGLang for GLM-5.3-Flash on two or four RTX PRO 6000 Blackwell GPUs (SM120), with TP2 W4A16 and TP4 NVIDIA NVFP4 profiles.
huoji120 / QWEN-EXO-booster
QWEN-BASE model-native memory and continuous learning system,base on SGLang
intel / gpu-ai-skills
Agent Skills for running, benchmarking, and profiling Hugging Face models on Intel GPUs — PyTorch, vLLM-XPU, and SGLang-XPU
knapcio / GLM-5.3-Flash-4x-DGX-Spark-TP4
GLM-5.3-Flash NVFP4 on 4x NVIDIA DGX Spark: vLLM TP4, DFlash2 adaptive draft, measured FP8 vs NVFP4 vs EXL3 vs SGLang
ormandj / sglang-deepseek-v4-flash-sm120
Reproducible SGLang image for DeepSeek-V4-Flash-0731 on SM120
pangoleen / qwen3.8-27b-dgx-spark-dflash2
Qwen3.8-27B on one DGX Spark: SGLang + DFlash2 recipe, engine image, benchmarks, data and charts. By @redp314
DocPang / qwen38-k100ai-int8-optimization
Qwen3.8-27B INT8/W8A8 optimization for Hygon K100AI: SGLang, TP1/TP4, DFlash2 and long-context Agent workloads
dg1kjd / sglang-sxm2
Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, MiniMax-H3.
luxingcom / LuZ-0.1.7-DeepSeek-v4.1-Flash-DGXspark-TP4-Ring
DeepSeek-V4.1-Flash on 4x DGX Spark TP4 switchless RoCE ring - SGLang production recipe, 600K context, DSpark MTP, benchmark archive
darkdatter / gb10-repo
Qwen3.8-27B on DGX Spark (GB10): SGLang + NVFP4 + DFlash2. Measured recipe, benchmark harness, and the traps that cost real time.