ztxz16 / fastllm

fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

intel / neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

intel / auto-round

PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度

dnv2003 / v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

DominikBucko / qwen38-flash-next-2x3090

Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).

point-4d / Point4D

Point4D: Long-range 4D Motion Reconstruction (NeurIPS 2026 Spotlight)

alesha-pro / atlas

Interactive canvas for taking an LLM apart tensor by tensor: measured INT8/INT4/FP8 error, distributions, spectra and outlier channels for every weight tensor

ForeverBlue816 / PrismQuant

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

azampatti / Qwen3.8-Flash-Next-Int4-FAST

Recipe for the fastest 4bit qwen3.8FlashNext

sybil-solutions / qwen-serve

Fastest validated sglang recipe for Qwen3.8-27B (AWQ-INT4 + DSpark) on RTX 3090s — 1×/2×/4× GPU presets, image+video, full benchmarks

0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.

MikeCaldera / intel-arc-pro-b70-qwen38-vllm

Reproducible Qwen3.8-27B GPTQ INT4 + MTP4 inference on Intel Arc Pro B70 using vLLM XPU — 84.65 tok/s median.

tlack / babytalk

Optimized ESP32-S3/P4 fully offline speech to text and text to speech system for Micropython and AtomVM, featuring a finetuned STT model for noisy environments, plus a data generation/field capture/model finetuning framework. Includes high performance 4/8bit int kernels for Xtensa/RiscV PIE vector units

RightNow-AI / local-kimi

Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.

sudoingX / dgx-spark-ling

Serve the official Ling-3.0-flash INT4 on one NVIDIA DGX Spark, 38.7 tok/s, faster than the community GGUF. Recipe, watchdog, benchmarks.