MaryCache / imozuru

知らない分野を、原文にたどりつくまで芋づる式に掘る Claude Code のスキル

kittycache9 / OpenSA2

A native Sonic Adventure 2 port by static recompilation for Linux, Windows, and Android. Experimental source preview.

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

BlinkDL / RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

LMCache / LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

mahmoud / boltons

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

bojieli / ai-infra-book

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

midoks / mdserver-web

Simple Linux Panel

morphik-org / morphik-core

Open-source multimodal retrieval engine (Morphik Core)

jazzband / django-redis

Full featured redis cache backend for Django.

django-compressor / django-compressor

Compresses linked and inline javascript or CSS into a single cached file.

tkem / cachetools

Extensible memoizing collections and decorators

syv-ai / HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

benavlabs / FastAPI-boilerplate

An extendable async API using FastAPI, Pydantic V2, SQLAlchemy 2.0, PostgreSQL and Redis.

ovg-project / kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

datawhalechina / zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

requests-cache / requests-cache

Persistent HTTP cache for python requests

vipshop / cache-dit

A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for DiTs.

brainix / pottery

Redis for humans. 🌎🌍🌏

NVIDIA / kvpress

LLM KV cache compression made easy

Zefan-Cai / R-KV

[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

redis / memtier_benchmark

NoSQL Redis and Memcache traffic generation and benchmarking tool.

tox-dev / platformdirs

A Python package for determining platform-specific directories (e.g. user data, config, cache, logs). Handles the differences between macOS, Windows, Linux/Unix, and Android so you don't have to.

aio-libs / async-lru

Simple LRU cache for asyncio

pallets-eco / flask-caching

A caching extension for Flask

jjang-ai / vmlx

vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm

pinterest / pymemcache

A comprehensive, fast, pure-Python memcached client.

thu-nics / C2C

[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"

mapproxy / mapproxy

MapProxy is a tile cache and WMS proxy

python-cachier / cachier

Persistent, stale-free, local and cross-machine caching for Python functions.

alisaifee / limits

Rate limiting using various strategies and storage backends such as redis, memcached & mongodb

jaylfc / taOS

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).

FutureMLS-Lab / OSCAR

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization

zengxiao-he / tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

psf / cachecontrol

The httplib2 caching algorithms packaged up for use with requests.

sybil-solutions / glm-flash-lite

GLM-5.3-Flash EXL3 on one 24 GB RTX 3090 + DDR4: elastic GPU expert cache, zero-copy experts, AVX2 CPU tier, OpenAI API

UditAkhourii / cdaf

CDAF (Cached Descriptive Asset Files) - open sidecar format for video so AI agents stop re-analyzing the same footage. Spec, CLI, agent skill, reproducible benchmark.

255308153 / CtxGuard

面向 AI Agent 的高并发、低延迟上下文治理与 Prompt Cache 守护网关 (Tree-sitter AST / Tool Delta / 50%~80% Token 削减)

lordx64 / phantom-kv

Refusal removal as a loadable KV-cache graft — zero weight modification, no refusal-direction projection, fully reversible.

jpezzulli / sglang-rtxpro6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

0xSero / deepseek-v4-flash-0731-spark-sparkinfer

Pinned one-DGX-Spark Docker recipe for DeepSeek V4 Flash with EXL3, SparkInfer, and 262K NVFP4 MLA KV cache

OnlyTerp / toolrush

Kill the tool-call tax: harness tool latency back below model TPS. Fast local lanes, persistent pools, session caches. Scratch lab turned real.

chadhurley25075-png / pd-bridge

Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

T8mars / comfyui-minimax-h3-blockcache-T8

—

sybil-solutions / dsv41-flash-offload

DeepSeek-V4.1-Flash EXL3 on one 24 GB RTX 3090 + DDR4 + NVMe: staged-DMA prefill, AVX2 CPU expert tier, elastic VRAM expert cache, Engram on disk, OpenAI API

apimart-22w / claude-opus-5-api

Claude Opus 5 API (claude-opus-5): model id, per-million-token pricing, cache write and cache read rates, streaming and Anthropic-style requests through an OpenAI-compatible gateway.

RH-RunningHub / MiniMax-H3-MultiGPU-Lightning

MiniMax-H3 multi-GPU inference acceleration: ~12x on 8x RTX 6000D (step distillation + SageAttention2 + Cache-DiT + torch.compile, TP2+Ulysses4 on sglang)

duckyshell / ComfyUI-MiniMaxH3-FirstBlockCache

Lightweight FirstBlockCache acceleration node for native ComfyUI MiniMax H3

deepmemteam / deepmem

Drop-in AI memory layer with 2x faster retrieval and 10x lower cost. Fully compatible with Mem0 API. Migrate in 5 minutes without any code changes. Self-host for free.

52Hz-kiko / mini-vllm-qwen3-serving

A Mini-vLLM optimization project for Qwen3-32B TP=2 serving, Prefix Cache, and Triton PagedAttention.

professorpalmer / mirai-s-ada

Mirai S (Qwen3.8-27B-S trellis GGUF) on a 12 GB RTX 4070: 262k q8_0 tiered cache, MTP drafting at every depth, harness-proofing, the layer; receipts.

rarf / hermes-quota-plugin

Per-provider quota and rate limits in Hermes Desktop: a status-bar chip, a docked pane, the /quota command and a CLI — fed from a local cache, with no network I/O in the UI.

eryflow / telos-sdk

TELOS SDK: a cache-aware prompt protocol and gateway for portable agent context.

lihaoyun6 / ComfyUI-MiniMaxH3-Cache

A better cache node for MiniMax H3 / 专为 MiniMax H3 优化的缓存加速节点

xzf-thu / BreezeTTS2_Mac_Streaming

Real-time Breeze-TTS-2 streaming on Apple Silicon — RTF from ~4 down to ~1. mlx-audio 0.5.1's depth decoder recomputes each frame's existing acoustic codes instead of reusing its KV cache. Adds intra-frame KV reuse, one CPU read-back per frame, an optional compiled whole-frame path, and RTF/underflow stats.

SalesforceAIResearch / Random-Attention

Code for the paper "Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning" https://arxiv.org/abs/2609.03430

ibrews / unreal-custodian

Reclaim the Intermediate, Binaries and cache folders your Unreal Engine projects and engines quietly hoard — without touching anything you authored.

cachemiao / m3u-studio

自托管 IPTV 频道编辑器：多列表管理、M3U/TXT 订阅、台标库、EPG 与自定义短链接。

Elevatormusic / hermes-warm-compaction

Hermes Agent context engine: the main model writes the compaction handoff on its cached prompt prefix