halt95 / merlin-engine

git clone https://github.com/halt95/merlin-engine.git

vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 184 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.