nopnoping / micro-vllm
A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
nopnoping / micro-vllm
NaiveAI-Labs / NaiveRT
NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.
alphastorm / omp-ninfer
Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.
moonlin1213 / cove-book-forge-mcp
Turn PDF and EPUB books into cached AI analysis, Obsidian notes, and installable Codex/Claude Agent Skills through an open MCP server.
Qin-sor / agent-context-lab
A minimal coding-agent harness built to measure what context management actually costs: constraint survival after compaction (8%→83%), default-FAIL verification against false completions, and KV-cache prefix design (21%→90%). Stdlib-only, real deepseek-chat runs
T8mars / Comfyui-Qwen-Image-2.1-BlockCache-T8
—
vincenzo-afk / Zen-2
A modern 124M-parameter decoder-only LLM built from scratch. RoPE · RMSNorm · SwiGLU · GQA · KV Cache · BPE Tokenizer · Full training pipeline. Research-grade, production-quality.
supElement / ComfyUI_MinimaxH3_AutoContext
A long video generation solution that breaks through the memory barrier: automatic segmented inference + inter-segment Latent anchoring, allowing MiniMax H3 to generate infinitely continuous long videos with limited video memory.
drowzeys / keys-SM121-Optimized-MiniMax-H3-Nvidia-Sol-Engine-Kijai-SolAttn_Triton-Single-DGX-Spark
MiniMax-H3 33B audio+video DiT accelerated 1.54x on a single NVIDIA DGX Spark (GB10, sm_121): NVIDIA Sol-Engine FirstBlockCache + batched VAE decode ported to single-GPU ComfyUI, Sol-Attn patched for compute 12.1, kijai Triton INT8/TMA kernels. Benchmarks, charts, one-shot installer, agent recipe.
CacheCarti / Crypto-Strategies
A repository of strategies I tested against 7 stage validation on 6 months of data
bunnyciaa / A12-A13-Hello-Screen-Bypass-with-SN-change-2
A12/A13 Hello screen bypass with SN change (FE00 activation record, gestalt CacheData patch, activation restore)
kadenball / qwen38-27b-rtx3060-dcfr
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.
nisesen / inferenceinquire
Real-time monitoring for llama.cpp inference servers: a read-only host agent plus a web dashboard for hardware, health and inference telemetry (prefill/decode vs context depth, slot occupancy, KV cache, GPU and PCIe).
SmetDenis / openwebui-token-usage-display
A filter plugin for Open WebUI that shows detailed token, context, timing and cost statistics below each AI response: token counts (input/output/total, reasoning, cached, audio), context-window utilization, generation time, tokens/second and message/chat cost.
FemtoRhythm / pyramis-l3d
Pyramis-L3D: hierarchical sparse attention research prototype (code only; weights on Hugging Face)
Tenosra / Proxima
Proxima lets existing GPUs serve 4x more concurrent requests