CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent

RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。

kvmem / kvmem

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

JakeATX / llamAmpere

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

1115714829 / llama.cpp-v100

llama.cpp optimized for NVIDIA Tesla V100 (SM70): large MoE Qwen3.8-Flash-Next with MTP (prefill 3464 tok/s at 200K context, 6x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 225 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving.

qhfan / FlashPrefillv2

FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving