supermochi64 / sm64-plaque-decode
Decoding the Super Mario 64 courtyard plaque ("L is real 2401"): font identification, forward model of the 1996 texture, line-by-line decoding, report and full reproduction code.
supermochi64 / sm64-plaque-decode
eagle-hq / EAGLE-4
Official Inference Code of EAGLE-4
NVIDIA / Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
raullenchai / Rapid-MLX
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4ร faster than Apple's MLX (mlx-lm) on the same weights.
youssofal / MTPLX
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.
syv-ai / HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
dphnAI / sonar
Large-scale LLM inference engine
Tencent / AngelSlim
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
nilearn / nilearn
Machine learning for NeuroImaging in Python
sgl-project / SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
meta-pytorch / torchcodec
Media decoding and encoding for PyTorch: videos, images, and audio, on CPU and GPU.
facebookresearch / brain2qwerty
Non-invasive decoding of typed sentences from MEG and EEG brain recordings using a convolutional encoder, transformer, and character-level language model.
StayLameBro / backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
vllm-project / speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
sstklen / trump-code
๐ AI decoding Trump's posts ร stock market | AI ่งฃ็ขผๅทๆฎๆจๆ ร ็พ่ก | AIใงใใฉใณใๆ็จฟรๆ ชๅผๅธๅ ดใ่งฃ่ชญ โ 31.5M models, 61.3% hit rate, open source
ARahim3 / mlx-dspark
Up to 4ร faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding โ Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.
ARahim3 / kaggle-tpu-lab
Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.
decodingai-magazine / building-a-coding-agent-from-scratch-course
Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.
zengxiao-he / tessera
From teacher to tiles โ a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
mkst / zte-config-utility
Scripts for decoding/encoding config.bin for ZTE routers
Yinsongxu / LLM2Jev
Turn local language models into Jev-style structured decision models. Get results from text and images with prefill aloneโno token-by-token decoding required.
malevrigns / agent-jev
AgentJev-0.6B - a fast 'System One' decision model for AI Agents: feed it any unstructured state (diffs, traces, logs) and structured questions, get calibrated probability distributions back in one ~50ms forward pass. Zero output-token decoding.
tritant / ComfyUI_MiniMax_H3_Extender
ComfyUI node for MiniMax H3 that chains multiple video clips with motion context, disk caching, dynamic image references, audio reference support, and seamless final video/audio decoding.
Tencent / AngelSpec
A unified, torch-native training framework for MTP and block-parallel speculative decoding.
julianmb / q38rocm
Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.
hasso5703 / dgx-spark-qwen38
One command turns a DGX Spark or ASUS Ascent GX10 into a private AI server: seven measured, pinned AI lanes (text up to 1M tokens, images, video with sound), run from a local cockpit on a laptop or a phone. Nothing leaves the box.
Nathanw1014 / strix-halo-llamacpp
Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.
SergiioB / intel-arc-pro-b70-inference-cookbook
Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!
dnv2003 / v100-skinny
Hand-written NVFP4 W4A16 CUDA kernels for Volta
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
ZinYY / AdaFlash
[COLMW'26 Spotlight] AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Jalen-Brunson / ComfyUI-MiniMax-H3-PDD-Acc
Official MiniMax-H3 8-step PDD Acc LoRAs (alibaba-pai) in ComfyUI: LoRA + parallel-decoding head bank, euler on trained sigmas, audio+video in 8 steps
peonist-ai / halogen-server
The fastest way to run Qwen3.8 27B on Strix Halo (gfx1151)
DominikBucko / qwen38-flash-next-2x3090
Qwen3.8-Flash-Next on 2ร RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill ยท 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill ยท 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
allover326 / deepseek-v4-cmp170hx
DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallelism, which vLLM does not support upstream.
jayleaton / glm53-tensorfold-spark
GLM-5.3-Flash (abliterated EXL3) on 2x NVIDIA DGX Spark with the TensorFold engine: 1.8x faster decode than vLLM, 4x256k concurrent threads, byte-exact speculative decoding. Work in progress.
zhengzihaoPKU / KERV
Code of paper "KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models" accepted by DAC 2026.
htobty / ReadyLLM
Deploy, monitor, and tune LLM through a visual interface โ plus image / video generation on top of ComfyUI.
hsj576 / virtual-ai-infra-team
An open-source virtual AI Infra team that discovers, benchmarks, and safely upgrades local LLM inferenceโwith independent quality gates, a stable OpenAI-compatible API, and automatic rollback.
dominique1993 / Crypto-Whale-Tracker
Real-time Ethereum whale tracker with WebSocket transaction streaming, ETH/ERC-20 decoding, and instant Telegram alerts above a configurable USD threshold. Wallet labels, counterparty analysis, smart-money PnL, exchange flows, SQLite storage, CSV/XLSX/JSON export, auto-reconnect, and cross-platform Python support.
julianmb / halofpx
High-performance Lemonade alternative for AMD Strix Halo & Radeon โ latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.
professorpalmer / bonsai-ada-surgery
Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.
Morrowmake / glm53-flash-cmp170hx-recipe
GLM-5.3-Flash on 4ร NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, one-command container install
groundingpi / GroundAnything
A 4B diffusion grounding model for fast, precise visual evidence extraction. Unifies diverse grounding tasks through parallel decoding, with optional self-speculative decoding delivering 4.51x autoregressive throughput on COCO.
Ch3nOff / dual-loop-controller
Hardware-aligned latent deliberation framework decoupling System 2 continuous reasoning from System 1 autoregressive decoding.
samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
MiaAI-Lab / Qwen3.8-27B-RTX-6000-PRO-SGLang-DSpark
Qwen3.8-27B on SGLang for NVIDIA RTX PRO 6000 (96GB, SM120): DSpark speculative decoding, full 256K context, 8 concurrent requests
0xBakeer / TandemLLM
An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 โ the only Qwen3.8-27B build that can serve it under vLLM.
0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark
Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness โ including why the quantization advantage collapses to +0.2% by c16.
alphastorm / omp-ninfer
Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.
KejiaZhang-Robust / CoVeil
[EMNLP Findings 2026] Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding
mbzuai-oryx / ParallelTubeDecoding
โ
decodingai-magazine / llm-wiki-workshop
Learn to design an LLM wiki from first principles. Used as agent memory. Workshop with 3 exercises, open-source code, video, and slides. No coding skills required.
hikarioyama / ParallelHue
Truthful scheduler-step and streaming visualization for local LLM inference
kadenball / qwen38-27b-rtx3060-dcfr
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.
dg1kjd / sglang-sxm2
Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, MiniMax-H3.
fufuchiu / speechloom
End-to-end speech model research: waveform encoding, causal decoding, codec tokens, joint losses and streaming.
StarWorkshop / wavrail
Streaming speech recognition toolkit: log-mel features, CTC greedy and prefix-beam decoding, transducer-style frame-synchronous emission, forced alignment, Chinese/English CER-WER metrics, and a chunked streaming pipeline with latency accounting. NumPy core with a CPU-only torch extra; trainable demos on synthetic audio, fully offline.