eagle-hq / EAGLE-4

Official Inference Code of EAGLE-4

syv-ai / HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

supermochi64 / sm64-plaque-decode

Decoding the Super Mario 64 courtyard plaque ("L is real 2401"): font identification, forward model of the 1996 texture, line-by-line decoding, report and full reproduction code.

vllm-project / speculators

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

StayLameBro / backburner

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

raullenchai / Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

youssofal / MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

jpezzulli / sglang-rtxpro6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

Tencent / AngelSlim

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

facebookresearch / brain2qwerty

Non-invasive decoding of typed sentences from MEG and EEG brain recordings using a convolutional encoder, transformer, and character-level language model.

ARahim3 / kaggle-tpu-lab

Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.

decodingai-magazine / building-a-coding-agent-from-scratch-course

Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.

Yinsongxu / LLM2Jev

Turn local language models into Jev-style structured decision models. Get results from text and images with prefill alone—no token-by-token decoding required.

malevrigns / agent-jev

AgentJev-0.6B - a fast 'System One' decision model for AI Agents: feed it any unstructured state (diffs, traces, logs) and structured questions, get calibrated probability distributions back in one ~50ms forward pass. Zero output-token decoding.

hasso5703 / dgx-spark-qwen38

One command turns a DGX Spark or ASUS Ascent GX10 into a private AI server: seven measured, pinned AI lanes (text up to 1M tokens, images, video with sound), run from a local cockpit on a laptop or a phone. Nothing leaves the box.

professorpalmer / bonsai-ada-surgery

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

groundingpi / GroundAnything

A 4B diffusion grounding model for fast, precise visual evidence extraction. Unifies diverse grounding tasks through parallel decoding, with optional self-speculative decoding delivering 4.51x autoregressive throughput on COCO.

decodingai-magazine / llm-wiki-workshop

Learn to design an LLM wiki from first principles. Used as agent memory. Workshop with 3 exercises, open-source code, video, and slides. No coding skills required.

dphnAI / sonar

Large-scale LLM inference engine

nilearn / nilearn

Machine learning for NeuroImaging in Python

sgl-project / SpecForge

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

meta-pytorch / torchcodec

Media decoding and encoding for PyTorch: videos, images, and audio, on CPU and GPU.

sstklen / trump-code

🔐 AI decoding Trump's posts × stock market | AI 解碼川普推文 × 美股 | AIでトランプ投稿×株式市場を解読 — 31.5M models, 61.3% hit rate, open source

ARahim3 / mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

zengxiao-he / tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

mkst / zte-config-utility

Scripts for decoding/encoding config.bin for ZTE routers

tritant / ComfyUI_MiniMax_H3_Extender

ComfyUI node for MiniMax H3 that chains multiple video clips with motion context, disk caching, dynamic image references, audio reference support, and seamless final video/audio decoding.

Tencent / AngelSpec

A unified, torch-native training framework for MTP and block-parallel speculative decoding.

julianmb / q38rocm

Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

Nathanw1014 / strix-halo-llamacpp

Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.

SergiioB / intel-arc-pro-b70-inference-cookbook

Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!

dnv2003 / v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

Blackwellboy / model-serving-minefield

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

ZinYY / AdaFlash

[COLMW'26 Spotlight] AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

Jalen-Brunson / ComfyUI-MiniMax-H3-PDD-Acc

Official MiniMax-H3 8-step PDD Acc LoRAs (alibaba-pai) in ComfyUI: LoRA + parallel-decoding head bank, euler on trained sigmas, audio+video in 8 steps

peonist-ai / halogen-server

The fastest way to run Qwen3.8 27B on Strix Halo (gfx1151)

DominikBucko / qwen38-flash-next-2x3090

Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).

allover326 / deepseek-v4-cmp170hx

DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallelism, which vLLM does not support upstream.

jayleaton / glm53-tensorfold-spark

GLM-5.3-Flash (abliterated EXL3) on 2x NVIDIA DGX Spark with the TensorFold engine: 1.8x faster decode than vLLM, 4x256k concurrent threads, byte-exact speculative decoding. Work in progress.

zhengzihaoPKU / KERV

Code of paper "KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models" accepted by DAC 2026.

htobty / ReadyLLM

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

hsj576 / virtual-ai-infra-team

An open-source virtual AI Infra team that discovers, benchmarks, and safely upgrades local LLM inference—with independent quality gates, a stable OpenAI-compatible API, and automatic rollback.

dominique1993 / Crypto-Whale-Tracker

Real-time Ethereum whale tracker with WebSocket transaction streaming, ETH/ERC-20 decoding, and instant Telegram alerts above a configurable USD threshold. Wallet labels, counterparty analysis, smart-money PnL, exchange flows, SQLite storage, CSV/XLSX/JSON export, auto-reconnect, and cross-platform Python support.

julianmb / halofpx

High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.

Morrowmake / glm53-flash-cmp170hx-recipe

GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, one-command container install

Ch3nOff / dual-loop-controller

Hardware-aligned latent deliberation framework decoupling System 2 continuous reasoning from System 1 autoregressive decoding.

samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.

MiaAI-Lab / Qwen3.8-27B-RTX-6000-PRO-SGLang-DSpark

Qwen3.8-27B on SGLang for NVIDIA RTX PRO 6000 (96GB, SM120): DSpark speculative decoding, full 256K context, 8 concurrent requests

0xBakeer / TandemLLM

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.

0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.

0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.

alphastorm / omp-ninfer

Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.

KejiaZhang-Robust / CoVeil

[EMNLP Findings 2026] Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding

mbzuai-oryx / ParallelTubeDecoding

—

hikarioyama / ParallelHue

Truthful scheduler-step and streaming visualization for local LLM inference

kadenball / qwen38-27b-rtx3060-dcfr

RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.

dg1kjd / sglang-sxm2

Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, MiniMax-H3.

fufuchiu / speechloom

End-to-end speech model research: waveform encoding, causal decoding, codec tokens, joint losses and streaming.

StarWorkshop / wavrail

Streaming speech recognition toolkit: log-mel features, CTC greedy and prefix-beam decoding, transducer-style frame-synchronous emission, forced alignment, Chinese/English CER-WER metrics, and a chunked streaming pipeline with latency accounting. NumPy core with a CPU-only torch extra; trainable demos on synthetic audio, fully offline.

jnardiello / GLM-5.3-Flash-FP8-4-DGX-Spark-Switchless

GLM-5.3-Flash (FP8) + DFlash2 speculative decoding on 4x NVIDIA DGX Spark / ASUS GX10: vLLM TP4, 256K context, switchless ConnectX-7 ring with patched NCCL