tigerless-labs / cost-xray
See what Claude Code and Codex actually send to the API — and what each part costs.
tigerless-labs / cost-xray
MaryCache / imozuru
知らない分野を、原文にたどりつくまで芋づる式に掘る Claude Code のスキル
bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
ovg-project / kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
cactus-compute / needle
Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
syv-ai / HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
jundot / omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
datawhalechina / zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
cocacola-lab / Recursia
Recursive self-improvement for robot manipulation, with real-robot experiments and shared executable skills.
LMCache / LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
Elevatormusic / hermes-warm-compaction
Hermes Agent context engine: the main model writes the compaction handoff on its cached prompt prefix
moonlin1213 / cove-book-forge-mcp
Turn PDF and EPUB books into cached AI analysis, Obsidian notes, and installable Codex/Claude Agent Skills through an open MCP server.
kvcache-ai / ktransformers
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
tonquer / picacg-qt
哔咔漫画, PicACG comic PC client(Windows, Linux, MacOS)
midoks / mdserver-web
Simple Linux Panel
vipshop / cache-dit
A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for DiTs.
NVIDIA / kvpress
LLM KV cache compression made easy
pallets-eco / flask-caching
A caching extension for Flask
ARahim3 / kaggle-tpu-lab
Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.
jaylfc / taOS
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).
OnlyTerp / toolrush
Kill the tool-call tax: harness tool latency back below model TPS. Fast local lanes, persistent pools, session caches. Scratch lab turned real.
caca2331 / finesub
Pipeline that turns audio/video into Chinese subtitles: vocal separation → VAD/ASR → optional LLM correction, translation, and knowledge-assisted polish.
rarf / hermes-quota-plugin
Per-provider quota and rate limits in Hermes Desktop: a status-bar chip, a docked pane, the /quota command and a CLI — fed from a local cache, with no network I/O in the UI.
ibrews / unreal-custodian
Reclaim the Intermediate, Binaries and cache folders your Unreal Engine projects and engines quietly hoard — without touching anything you authored.
NaiveAI-Labs / NaiveRT
NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.
kittycache9 / OpenSA2
A native Sonic Adventure 2 port by static recompilation for Linux, Windows, and Android. Experimental source preview.
BlinkDL / RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
zilliztech / GPTCache
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
mahmoud / boltons
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
joblib / joblib
Computing with Python functions.
morphik-org / morphik-core
Open-source multimodal retrieval engine (Morphik Core)
jazzband / django-redis
Full featured redis cache backend for Django.
django-compressor / django-compressor
Compresses linked and inline javascript or CSS into a single cached file.
tkem / cachetools
Extensible memoizing collections and decorators
benavlabs / FastAPI-boilerplate
An extendable async API using FastAPI, Pydantic V2, SQLAlchemy 2.0, PostgreSQL and Redis.
fabiocaccamo / django-admin-interface
:superhero: :zap: django's default admin interface with superpowers - customizable themes, popup windows replaced by modals and many other features.
fabiocaccamo / python-benedict
:blue_book: dict subclass with keylist/keypath support, built-in I/O operations (base64, csv, html, ini, json, pickle, plist, query-string, toml, xls, xml, yaml), s3 support and many utilities.
requests-cache / requests-cache
Persistent HTTP cache for python requests
brainix / pottery
Redis for humans. 🌎🌍🌏
Zefan-Cai / R-KV
[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
redis / memtier_benchmark
NoSQL Redis and Memcache traffic generation and benchmarking tool.
tox-dev / platformdirs
A Python package for determining platform-specific directories (e.g. user data, config, cache, logs). Handles the differences between macOS, Windows, Linux/Unix, and Android so you don't have to.
aio-libs / async-lru
Simple LRU cache for asyncio
jjang-ai / vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
pinterest / pymemcache
A comprehensive, fast, pure-Python memcached client.
fabiocaccamo / django-treenode
:deciduous_tree: probably the best abstract model/admin for your tree based stuff.
nipy / nibabel
Python package to access a cacophony of neuro-imaging file formats
thu-nics / C2C
[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"
ComparativeGenomicsToolkit / cactus
Official home of genome aligner based upon notion of Cactus graphs
mapproxy / mapproxy
MapProxy is a tile cache and WMS proxy
python-cachier / cachier
Persistent, stale-free, local and cross-machine caching for Python functions.
fabiocaccamo / django-colorfield
:art: color field for django models with a nice color-picker in the admin.
alisaifee / limits
Rate limiting using various strategies and storage backends such as redis, memcached & mongodb
fabiocaccamo / django-extra-settings
:gear: config and manage typed extra settings using just the django admin.
FutureMLS-Lab / OSCAR
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
zengxiao-he / tessera
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
fabiocaccamo / django-maintenance-mode
:construction: :hammer_and_wrench: shows a 503 error page when maintenance-mode is on.
psf / cachecontrol
The httplib2 caching algorithms packaged up for use with requests.
tritant / ComfyUI_MiniMax_H3_Extender
ComfyUI node for MiniMax H3 that chains multiple video clips with motion context, disk caching, dynamic image references, audio reference support, and seamless final video/audio decoding.