Mengqi-Lei / texglot
Translate arXiv papers and LaTeX projects with your own LLM API, preserving equations, citations, and document structure. Generate translated PDFs and editable source through a local app or batch CLI.
Mengqi-Lei / texglot
ArtificialAnalysis / aa-agentperf-local
Benchmark local LLM serving by replaying real agent trajectories
NoizAI / YuE2-Turbo
Fast, concurrent inference for YuE2. Same model and recipe: 1.68× faster per song, 3.31× more songs per GPU.
htobty / ReadyLLM
Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.
52Hz-kiko / mini-vllm-qwen3-serving
A Mini-vLLM optimization project for Qwen3-32B TP=2 serving, Prefix Cache, and Triton PagedAttention.
lin1111-1 / academic-integrity-rewrite
Codex skill for evidence-preserving academic rewriting in Chinese and English.
led829155-droid / RecSys-MLops
Production-style end-to-end e-commerce recommendation platform with data engineering, MLOps, serving, governance, and observability on Kubernetes.
hsj576 / virtual-ai-infra-team
An open-source virtual AI Infra team that discovers, benchmarks, and safely upgrades local LLM inference—with independent quality gates, a stable OpenAI-compatible API, and automatic rollback.
ai-dynamo / aisimulate
AISimulate predicts LLM serving behavior and searches for strong deployment configurations offline, without bringing up a GPU serving cluster
sohu-mptc / FlashRec
FlashRec is a CUDA-graph engine for generative recommendation: wide beam search (3–5 SID steps, n=50–512+) over a trie-constrained catalog, in-process FP8 serving, and ranked beams on /v1/chat/completions.
Enntity / sparkglm
Fast, capable GLM-5.3-Flash FP4-Local on two DGX Sparks: opinionated Atlas serving and reproducible inference research.
thu-nics / TokenRouter
[NeurIPS'26] The official code implementation for paper "TokenRouter: Efficient Serving System for Token-Level LLM Routing"
Reederey87 / glm53-flash-exl3-2x-dgx-spark
GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode
0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark
Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.
openfabric-systems / simllm
Network-faithful simulation of LLM serving and training deployments
0xBakeer / Qwen3.8-27B-4-bit-on-a-single-DGX-Spark
Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.
anemoi-project / anemoi
Anemoi is a high-performance serving framework for transformer-based video generation models.
nevertoday / xxd-strip-ai-meta
Batch-remove AI provenance and image metadata with ExifTool. CLI + Agent Skill, preserving pixel data.
QuadraV-Speech / CosyVoice3Pro
Production-ready CosyVoice serving with Triton, reusable Speaker Registry, Public HTTP API and Web console.
RightNow-AI / local-kimi
Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.
rulith-dev / rulith-inference
Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.
songhai-dg / review-write
Evidence-preserving professional writing review and rewrite Skill for Chinese and English: review first, protect facts, deliver clean body.
othexmr / GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE
GLM-5.3-Flash NVFP4 on two or four DGX Sparks cabled as a ring, with no switch: the RiNGSiDE serving recipe
yuanzhy29 / PXDepth
PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation
dolf3131 / dgx-spark-llm-serving
Serving large LLMs on a single DGX Spark (121 GiB): Qwen3.8-Flash-Next on TensorFold/SparkFold EXL3 and vLLM NVFP4, Nemotron 3.5 Super VL on vLLM; what ran, what it measured, what did not work