RadixRootMind / RadixQuantVLA

RadixQuantVLA is a unified, Lego-like framework for post-training quantization, benchmarking, and deployment preparation of Vision-Language-Action models, integrating QuantVLA, Omega-QVLA, OpenVLA, UniVLA, StarVLA, GR00T, and Pi0.5 routes with standardized launchers and validation on LIBERO benchmarks.

htobty / ReadyLLM

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

Dreamer-Toby / STEPQuant

STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

Saganaki22 / ComfyUI-Breeze-TTS-2

Breeze TTS 2 nodes for ComfyUI: voice clone, voice design, and voice direction with bilingual (EN/ZH) speech, inline vocal events, INT8 ConvRot quantized builds, and full ComfyUI / AIMDO dynamic-VRAM integration.

a2307588073-arch / zynq-yolov3-tiny-accelerator

INT8 single-class YOLOv3-Tiny accelerator and OV5640-to-HDMI bare-metal demo for Xilinx Zynq-7000 XC7Z100, with PyTorch training and parameter-export tools. Topics yolov3-tiny, fpga, zynq-7000, xc7z100, verilog, vivado, vitis, pytorch, int8-quantization, object-detection, axi4-stream, ov5640, hdmi, embedded-systems, computer-vision

julianmb / halofpx

High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.

poisonxa16 / pxa

The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; benchmarks vs llama.cpp in the README.

Tylogi / MFQ

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

professorpalmer / bonsai-ada-surgery

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

alesha-pro / atlas

Interactive canvas for taking an LLM apart tensor by tensor: measured INT8/INT4/FP8 error, distributions, spectra and outlier channels for every weight tensor

headpiece747 / ninfer-5090-windows

Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context

ayutaz / sanoTTS-jp

559 K パラメータの日本語 TTS を ESP32-S3 で実時間合成。漢字かな交じり文の形態素解析・アクセント推定まで端末内で走る（M5Stack CoreS3 実機で確認）。推論は依存ゼロの C99、ブラウザ demo あり。arXiv:2608.21378 sanoTTS の日本語 clean-room 再実装。⚠️ コードは MIT ですが、配布モデルの重みは MIT ではありません（LICENSE-MODEL.md。出力に用途制限が伝播します）

ForeverBlue816 / PrismQuant

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

0xBakeer / TandemLLM

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.

SweiryDev / MicroCNN-TangNano20k

SystemVerilog SoC implementing MicroCNN accelerator on the Tang Nano 20K for BloodMNIST image classification.

chilly23 / RP2040-and-ESP32-AI

Run a 28.9M-parameter TinyLM on ESP32-S3 with an RP2040 OLED display node for fully local embedded AI inference.

0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.

paicat1 / Bonsai-27B-NInfer

Bonsai-27B x NInfer local deployment project (RTX 5080, ternary quantization)

Bellman281 / esp32-tinyLLM

Rust inference engine for a 28.9M-parameter PLE TinyLM on the ESP32-S3 — 9.4 tok/s, faster than the C implementation it was ported from

RadixRootMind / Radix-Embodied-Stack

Radix Embodied Stack: Universal infrastructure for VLA model quantization, inference, and deployment on domestic chips . Cross-model, cross-chip, cross-platform.

tlack / babytalk

Optimized ESP32-S3/P4 fully offline speech to text and text to speech system for Micropython and AtomVM, featuring a finetuned STT model for noisy environments, plus a data generation/field capture/model finetuning framework. Includes high performance 4/8bit int kernels for Xtensa/RiscV PIE vector units

drJhonatan00 / Game_Translation_Pipeline

A high-performance, offline game translation framework built in Python. Powered by CTranslate2 (INT8 quantization) and Helsinki-NLP neural models, this tool automates multi-language translation for game scripts while strictly protecting low-level syntax, variables, and memory pointers.

ronitgupta138 / Vectra-Core

⚡ In-Memory Vector Search Engine & HNSW ANN Graph in Java 21 Loom. Sub-millisecond KNN retrieval, 8-bit scalar quantization (SQ8), 16,000+ QPS.

PedroMarinhoDev / ComfyUI-HunyuanImage3

Run Tencent's HunyuanImage-3.0 (80B MoE) natively in ComfyUI on a 24 GB GPU: base, Instruct and Instruct-Distil, prompt rewriting, image editing, Spectrum speed-up.

sudoingX / dgx-spark-deepseek-v41

DeepSeek-V4.1-Flash on two NVIDIA DGX Sparks: EXL3 weights from the official checkpoint, upstream vLLM with our patches and decode kernels

RightNow-AI / local-kimi

Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.

Greninja9257 / LabLLM

A native macOS lab for teaching tiny language models to think — build the architecture, train the weights, and watch a small LLM emerge from scratch, locally on Apple Silicon with custom data, tokenizers, checkpoints, and MLX acceleration.

sauravsingla / MemVanta

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.

rostamlabs / rostam

Open-source vector database and sub-microsecond key-value store in one Go engine — embed it as a library, run it standalone, or replicate it across a Raft cluster. HNSW/IVF/Vamana indexes, quantization, hybrid dense+sparse and BM25 search, WASM stored procedures. Apache-2.0.

sepiablue-ai / ComfyUI-H3-X2-Stream

ComfyUI nodes for MiniMax H3: INT8 X2 VAE and asynchronous NVENC saving. 5-second 1088x1920 video with audio in about 60 s on an RTX 4070 12GB, about 30% faster than the FP16 X2 VAE.

ExVRAM / exvram-lab

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

yzfly / edgejev

离线可用的本地类型化决策：4 核 CPU 单题 15.6ms。Local & offline Jev / System One inference on CPU — ONNX + INT8, no torch at runtime. 支持 laya / kev / PlayJev

WestWaters / pollard-weights

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.

IST-DASLab / disaggregated-quantization

Disaggregated quantization official codebase

shyringo / deepseek-v4-flash-0731-in-c

Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, resident OpenAI-compatible API with function tools. | 在笔记本单颗 CPU 上本地运行原生 284B-A13B DeepSeek-V4-Flash-0731 大模型：纯 C，最低 8 GB 内存，无需 GPU，最优 TPOT 0.892 秒/token，支持模型常驻的 OpenAI 兼容接口与函数工具。

julianmb / haloq38flash

qwen3.8-flash-next on strix halo — 91g quant, 56 tok/s, 262k context, receipts included

vcruz305 / GLM-5.3-Flash-EXL3-K2-DGX-Spark-recipe

vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.

vcruz305 / vllm-exl3

Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights

kdrkdrkdr / faster-enhancer.c

Pure C implementation of FastEnhancer. 3.3x faster than the ONNX Runtime graph, same quality.

rayl15 / Godwit

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no dependencies.

harshitkhandelwal208 / hk

Local LLMs that run lean, load instantly and keep getting smarter. A from-scratch runtime + model format in Zig: zero-copy mmap, CPU/CUDA/Vulkan engines, OpenAI-compatible server, 2:4 sparsity, Net2Net growth, in-file version history, self-learning loops.

r0b0tlab / qwen38-exl3-dflash2

Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090