MemTensor / MemT-1.0
Official repository for MemT-1.0
MemTensor / MemT-1.0
PaddlePaddle / FlashMask
Efficient Mask-Aware Distributed Attention for Long-Context Training
CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent
RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。
open-compass / opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
rednote-machine-learning / RedKnot
Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
NVIDIA / kvpress
LLM KV cache compression made easy
microsoft / MInference
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
kvmem / kvmem
KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.
peonist-ai / halogen-flash-server
The fastest way to run Qwen3.8-Flash-Next on Strix Halo (gfx1151)
ARahim3 / kaggle-tpu-lab
Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.
canarybyte / veridrop
在线检测：veridrop.app｜AI API 中转站检测工具：Claude 中转站检测、OpenAI 中转站检测、Gemini 中转站检测，中转站真伪检测、长上下文验证、思维签名验证、中转站红黑榜，自托管开源。
grishahq / recursive-llm
Recursive Language Models for efficient long-context processing. Analyze 1M+ tokens by storing context in a Python REPL while reducing LLM token usage.
JakeATX / llamAmpere
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
jinhanbuilds / hoist-the-elephant
Re-anchor long conversations that drift through repeated local corrections.
samuelcardillo / glm-5.3-flash-2x-rtx-pro-6000-blackwell
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
wanqili857-byte / fictionforge
多智能体小说写作框架 | Multi-agent novel-writing framework — 每个角色都有自己的脑子,每一章都有质量门禁
1115714829 / llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70): large MoE Qwen3.8-Flash-Next with MTP (prefill 3464 tok/s at 200K context, 6x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 225 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving.
thaddeuswycliffe / glm-5-3-api-guide
A focused Argolink guide for GLM 5.3: the exact model ID, chat-compatible request, reasoning controls, long-context limits, and production error handling.
vdev-ctrl / Adaptive-Learnable-Hierarchical-Routing-
This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.
nibor1896 / Crow
Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen-Image. Runs on crow-nest or llama.cpp on one GPU, or any model via OpenRouter. Windows and Linux.
RinDig / cost-of-remembering
The Cost of Remembering: filesystem memory matches long-context accuracy on LongMemEval while reading 97% fewer tokens and costing 95% less. Harness, run data, 129 agent-built memories, and paper source.
satellitedown / fast-long-context-cinference
One-menu RTX 5090 setup for non-Swift Huihui Qwen3.8-27B NVFP4 with Cinference, MTP-10 and K8V4.
Veloresearch / velocity-local-services
Velocity Local Services: an app for running LLMs locally on Windows and Linux. Load any GGUF model and chat with web search, tools and schematics, let it use your computer by voice, and serve it to opencode, Hermes or any app through an OpenAI-compatible API and MCP. Context up to 32M tokens. No cloud.
DocPang / qwen38-k100ai-int8-optimization
Qwen3.8-27B INT8/W8A8 optimization for Hygon K100AI: SGLang, TP1/TP4, DFlash2 and long-context Agent workloads
dg1kjd / sglang-sxm2
Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, MiniMax-H3.
satellitedown / fast-long-context
Easy local setup for Qwen3.8-27B Abliterated: 256K context and measured 240-300 tokens/s generation on an RTX 5090.
qhfan / FlashPrefillv2
FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving
ZipengWu365 / TOMC
TOMC Memory: CPU context compression and task memory for Codex, Claude Code, Cursor, TRAE, WorkBuddy and other MCP clients.