CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent
RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。
CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent
mozilla-ai / llamafile
Distribute and run LLMs with a single file.
mw00 / project-maya
Project Maya - run GLM-5.3-Flash (321B MoE) and DeepSeek-V4.1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
feder-cr / jev
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
VecSzn / PolyStrata
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
Indras-Mirror / Xolotl
Xolotl (formerly Strata-DSV4.1): DeepSeek-V4.1-Flash, all 552B, on one gaming PC (RTX 4090 + 90 GB RAM) at ~27 tok/s, plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B. A heavily extended fork of Strata: MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.
JakeATX / llamAmpere
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
IIIIIllllIIIIIlllll / Qwenox
An inference framework customized for AI MAX 395, compatible with both Linux and Windows, and supports deploying only qwen3.8-flash-next. This project is for reference and learning purposes only. 为AI MAX 395定制的推理框架，Linux和Windows通用，只能部署qwen3.8-flash-next。本项目仅供参考学习使用。
Tylogi / MFQ
Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.
shawnshekari / strixite
LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): large MoE Qwen3.8-Flash-Next with MTP (prefill 3464 tok/s at 200K context, 6x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 225 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving.
sauravsingla / MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.