turboderp-org / exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

theroyallab / tabbyAPI

The official API server for Exllama. OAI compatible, lightweight, and fast.

Yamz-Labs / kyojin

Kyojin: the Yamz inference engine for AMD Strix Halo (ROCm, gfx1151), built on ExLlamaV3. Runs 300B-class MoE models on one 128 GB mini PC.

architectds / collabosm

A local app that rents a GPU in your own Colab account and serves Qwen3.8 on it: 27B on an A100-40G, or Flash-Next (125B-A6B MoE) on an A100-80G. The cost is shown before it starts; chat with images; Codex or any OpenAI client uses a fixed local /v1. Auto-stop, CU ledger, live prefill/decode. Windows, macOS, Linux.

vcruz305 / Qwen3.8-Flash-Next-EXL3-DGX-Spark-recipe

Serve Qwen3.8-Flash-Next (turboderp ExLlamaV3 pack) on one NVIDIA DGX Spark with exllamav3

vcruz305 / vllm-exl3

Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights

r0b0tlab / qwen38-exl3-dflash2

Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090