MaryCache / imozuru
知らない分野を、原文にたどりつくまで芋づる式に掘る Claude Code のスキル
MaryCache / imozuru
kittycache9 / OpenSA2
A native Sonic Adventure 2 port by static recompilation for Linux, Windows, and Android. Experimental source preview.
kvcache-ai / ktransformers
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
BlinkDL / RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
LMCache / LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
zilliztech / GPTCache
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
mahmoud / boltons
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
midoks / mdserver-web
Simple Linux Panel
morphik-org / morphik-core
Open-source multimodal retrieval engine (Morphik Core)
jazzband / django-redis
Full featured redis cache backend for Django.
django-compressor / django-compressor
Compresses linked and inline javascript or CSS into a single cached file.
tkem / cachetools
Extensible memoizing collections and decorators
syv-ai / HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
benavlabs / FastAPI-boilerplate
An extendable async API using FastAPI, Pydantic V2, SQLAlchemy 2.0, PostgreSQL and Redis.
ovg-project / kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
datawhalechina / zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
requests-cache / requests-cache
Persistent HTTP cache for python requests
vipshop / cache-dit
A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for DiTs.
brainix / pottery
Redis for humans. 🌎🌍🌏
NVIDIA / kvpress
LLM KV cache compression made easy
Zefan-Cai / R-KV
[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
redis / memtier_benchmark
NoSQL Redis and Memcache traffic generation and benchmarking tool.
tox-dev / platformdirs
A Python package for determining platform-specific directories (e.g. user data, config, cache, logs). Handles the differences between macOS, Windows, Linux/Unix, and Android so you don't have to.
aio-libs / async-lru
Simple LRU cache for asyncio
pallets-eco / flask-caching
A caching extension for Flask
jjang-ai / vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
pinterest / pymemcache
A comprehensive, fast, pure-Python memcached client.
thu-nics / C2C
[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"
mapproxy / mapproxy
MapProxy is a tile cache and WMS proxy
python-cachier / cachier
Persistent, stale-free, local and cross-machine caching for Python functions.
alisaifee / limits
Rate limiting using various strategies and storage backends such as redis, memcached & mongodb
jaylfc / taOS
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).
FutureMLS-Lab / OSCAR
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
zengxiao-he / tessera
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
psf / cachecontrol
The httplib2 caching algorithms packaged up for use with requests.
sybil-solutions / glm-flash-lite
GLM-5.3-Flash EXL3 on one 24 GB RTX 3090 + DDR4: elastic GPU expert cache, zero-copy experts, AVX2 CPU tier, OpenAI API
UditAkhourii / cdaf
CDAF (Cached Descriptive Asset Files) - open sidecar format for video so AI agents stop re-analyzing the same footage. Spec, CLI, agent skill, reproducible benchmark.
255308153 / CtxGuard
面向 AI Agent 的高并发、低延迟上下文治理与 Prompt Cache 守护网关 (Tree-sitter AST / Tool Delta / 50%~80% Token 削减)
lordx64 / phantom-kv
Refusal removal as a loadable KV-cache graft — zero weight modification, no refusal-direction projection, fully reversible.
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
0xSero / deepseek-v4-flash-0731-spark-sparkinfer
Pinned one-DGX-Spark Docker recipe for DeepSeek V4 Flash with EXL3, SparkInfer, and 262K NVFP4 MLA KV cache
OnlyTerp / toolrush
Kill the tool-call tax: harness tool latency back below model TPS. Fast local lanes, persistent pools, session caches. Scratch lab turned real.
chadhurley25075-png / pd-bridge
Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE
T8mars / comfyui-minimax-h3-blockcache-T8
—
sybil-solutions / dsv41-flash-offload
DeepSeek-V4.1-Flash EXL3 on one 24 GB RTX 3090 + DDR4 + NVMe: staged-DMA prefill, AVX2 CPU expert tier, elastic VRAM expert cache, Engram on disk, OpenAI API
apimart-22w / claude-opus-5-api
Claude Opus 5 API (claude-opus-5): model id, per-million-token pricing, cache write and cache read rates, streaming and Anthropic-style requests through an OpenAI-compatible gateway.
RH-RunningHub / MiniMax-H3-MultiGPU-Lightning
MiniMax-H3 multi-GPU inference acceleration: ~12x on 8x RTX 6000D (step distillation + SageAttention2 + Cache-DiT + torch.compile, TP2+Ulysses4 on sglang)
duckyshell / ComfyUI-MiniMaxH3-FirstBlockCache
Lightweight FirstBlockCache acceleration node for native ComfyUI MiniMax H3
deepmemteam / deepmem
Drop-in AI memory layer with 2x faster retrieval and 10x lower cost. Fully compatible with Mem0 API. Migrate in 5 minutes without any code changes. Self-host for free.
52Hz-kiko / mini-vllm-qwen3-serving
A Mini-vLLM optimization project for Qwen3-32B TP=2 serving, Prefix Cache, and Triton PagedAttention.
professorpalmer / mirai-s-ada
Mirai S (Qwen3.8-27B-S trellis GGUF) on a 12 GB RTX 4070: 262k q8_0 tiered cache, MTP drafting at every depth, harness-proofing, the layer; receipts.
rarf / hermes-quota-plugin
Per-provider quota and rate limits in Hermes Desktop: a status-bar chip, a docked pane, the /quota command and a CLI — fed from a local cache, with no network I/O in the UI.
eryflow / telos-sdk
TELOS SDK: a cache-aware prompt protocol and gateway for portable agent context.
lihaoyun6 / ComfyUI-MiniMaxH3-Cache
A better cache node for MiniMax H3 / 专为 MiniMax H3 优化的缓存加速节点
xzf-thu / BreezeTTS2_Mac_Streaming
Real-time Breeze-TTS-2 streaming on Apple Silicon — RTF from ~4 down to ~1. mlx-audio 0.5.1's depth decoder recomputes each frame's existing acoustic codes instead of reusing its KV cache. Adds intra-frame KV reuse, one CPU read-back per frame, an optional compiled whole-frame path, and RTF/underflow stats.
SalesforceAIResearch / Random-Attention
Code for the paper "Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning" https://arxiv.org/abs/2609.03430
ibrews / unreal-custodian
Reclaim the Intermediate, Binaries and cache folders your Unreal Engine projects and engines quietly hoard — without touching anything you authored.
cachemiao / m3u-studio
自托管 IPTV 频道编辑器：多列表管理、M3U/TXT 订阅、台标库、EPG 与自定义短链接。
Elevatormusic / hermes-warm-compaction
Hermes Agent context engine: the main model writes the compaction handoff on its cached prompt prefix