kvcache-ai / ktransformers
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
kvcache-ai / ktransformers
BlinkDL / RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
LMCache / LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
automatic-ripping-machine / automatic-ripping-machine
Automatic Ripping Machine (ARM) Scripts
foryourhealth111-pixel / Vibe-Skills
Intelligent Skill routing and workflow orchestration for AI agents — +21.12 pp reward, −29.6% tokens on SkillsBench with DeepSeekV4Flash-VE.
piskvorky / smart_open
Utils for streaming large files (S3, HDFS, gzip, bz2...)
rednote-machine-learning / RedKnot
Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
jorhelp / Ingram
网络摄像头漏洞扫描工具 | Webcam vulnerability scanning tool
syv-ai / HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
ovg-project / kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
pingcap / docs-cn
TiDB/TiKV/PD 中文文档
mdhiggins / sickbeard_mp4_automator
Automatically convert video files to a standardized format with metadata tagging to create a beautiful and uniform media library
datawhalechina / zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
CERT-Polska / drakvuf-sandbox
DRAKVUF Sandbox - automated hypervisor-level malware analysis system
kvesteri / sqlalchemy-utils
Various utility functions and datatypes for SQLAlchemy.
NVIDIA / kvpress
LLM KV cache compression made easy
Zefan-Cai / R-KV
[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
cenglin123 / SteganographierGUI
将文件隐写进MP4/MKV文件中 （Embed files into MP4/MKV files.）
jjang-ai / vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
egorsmkv / simple-django-login-and-register
An example of Django project with basic user functionality.
ikvk / imap_tools
Work with email by IMAP
jishnu-kv / WinUI-3-Apps-List
A curated collection of over 800+ apps that embrace the WinUI Design
thu-nics / C2C
[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"
emersonfelipesp / netbox-proxbox
Netbox Plugin for integration between Proxmox and Netbox
ganeti / ganeti
Ganeti is a virtual machine cluster management tool built on top of existing virtualization technologies such as Xen or KVM and other open source software.
jaylfc / taOS
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).
FutureMLS-Lab / OSCAR
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
zengxiao-he / tessera
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
abbbi / virtnbdbackup
Backup utility for Libvirt / qemu / kvm supporting incremental and differential backups + instant recovery (agentless).
vizart-vj / ComfyUI-MiniMax-H3-LongMedia
ComfyUI custom nodes for MiniMax H3 long-form video/audio generation with streamed Sol attention, compressed KV, adaptive VRAM guards, chunked MLP/final output, and SAFE long-sequence optimizations for limited VRAM.
kalkman-code / beamer
One keyboard and mouse for a Mac and a Windows PC over the local network, in either direction.
kvuong2711 / fix-anything
[ECCV 2026] FixAnything: 3D-Consistent Rendering Refinement via Video Generative Priors
drozbay / MaskVidExperiments
Video masking tools for ComfyUI: process a masked subject at high resolution inside a moving crop, then paste it back without jitter or visible seams
lordx64 / phantom-kv
Refusal removal as a loadable KV-cache graft — zero weight modification, no refusal-direction projection, fully reversible.
0xSero / deepseek-v4-flash-0731-spark-sparkinfer
Pinned one-DGX-Spark Docker recipe for DeepSeek V4 Flash with EXL3, SparkInfer, and 262K NVFP4 MLA KV cache
chadhurley25075-png / pd-bridge
Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
tonyd2wild / Qwen3.8-Flash-Next-NVFP4-DGX-Spark
Qwen3.8-Flash-Next (NVFP4) on DGX Spark in vLLM: one Spark 43.9 tok/s with our disk-backed n-gram table patch, staged gather and reduced-vocab MTP draft; TP2 SPEED 53.7; TP4 CONTEXT 9M KV pool. Launchers, patch, 40-prompt harness, KV ledger, credits. SGLang TP2 day-0 lane kept.
morpheuslord / PICOTTY
PicoTTY is a simple serial tty serial console or somewhat of a IP KVM cause my homelab was just proxmox so serial is all I need.
Rimcat-JA / blockvideo
日本語の台本を貼るだけでスライド・VOICEVOX音声・焼き込み字幕入りのMP4を出力するローカルWebアプリ
star7code / minimax-h3-chunk-star7
VRAM-efficient MiniMax H3 for ComfyUI: QKV/RoPE/MLP activation chunking, automatic OOM chunk reduction, and CK/SLA/Sol/Hybrid attention for SM75 and SM80+ GPUs.
professorpalmer / bonsai-ada-surgery
Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.
xzf-thu / BreezeTTS2_Mac_Streaming
Real-time Breeze-TTS-2 streaming on Apple Silicon — RTF from ~4 down to ~1. mlx-audio 0.5.1's depth decoder recomputes each frame's existing acoustic codes instead of reusing its KV cache. Adds intra-frame KV reuse, one CPU read-back per frame, an optional compiled whole-frame path, and RTF/underflow stats.
SalesforceAIResearch / Random-Attention
Code for the paper "Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning" https://arxiv.org/abs/2609.03430
beskvladimir-create / tsiolkovsky-papers
Machine-readable catalogue of fond 555 (K.E. Tsiolkovsky), Archive of the Russian Academy of Sciences: all 2,019 files, 51,008 scans, plus the retrieval pipeline.
ai-dynamo / kvcr
—
shi3z / deepseekv4.1-A100-custom
—
nopnoping / micro-vllm
A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
spicylemonade / compensated-magnet-ledger
Computational ledger for two room-temperature Luttinger-compensated magnet candidates, YBaMnFeO5 and KV[Cr(CN)6]: raw QE inputs/outputs, checker, re-runs
PeakVisionAI / PeakVisionOS-SDK
—
tonyd2wild / GLM-5.3-Flash-NVFP4-1M-KV-4x-DGX-Spark
GLM-5.3-Flash (320B MoE) at TP4 across four DGX Sparks, same day as the model drop: 36 tok/s, 1.26M-token FP8 KV pool, 262K context, MTP spec decode. First TP4 glm5_next outside B200 hardware. Full patched-image recipe + the GB10 memory study.
apimartgwbkkvm / reverse-engineered-api-ko
리버스와 공식 라우트를 한곳에. `-ext` 는 리버스, 나머지는 공식 할인 단가.AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
xmarre / ComfyUI-Sol-H3
Native MiniMax-H3 inference acceleration for ComfyUI, integrating Exact Runtime optimizations with Sana Sol-Attn, rectangular Q/KV attention, and composable support for VDN, Spectrum, Untwisting RoPE, Diff-Aid, and Flow mixed-grid workflows.
tonyd2wild / MiMo-V2.6-Flash-DGX-Spark-Recipe
MiMo-V2.6-Flash-RL on 2x NVIDIA DGX Spark (GB10): vLLM TP2 + DFlash + vision/audio, fp8 KV 300K, four fixes for the stock image, measured baselines
alphastorm / omp-ninfer
Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.
cnaf609itkvimah / sora-2-pro-api-ja
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
Qin-sor / agent-context-lab
A minimal coding-agent harness built to measure what context management actually costs: constraint survival after compaction (8%→83%), default-FAIL verification against false completions, and KV-cache prefix design (21%→90%). Stdlib-only, real deepseek-chat runs
apimarts4kv3xigz / api-gateway-pricing-2026-fr
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.
apimarts4kv3xigz / api-gateway-pricing-2026-ja
AI API gateway / relay: 307 models behind one OpenAI-compatible key. image2.5 $0.0085/image, Seedance 2.0 Mini $0.01056/sec, LLM from $0.0228/M tokens. USD settlement, pay-as-you-go, $1 minimum top-up.