Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
THE FIRST DLSS 5 MANAGER
使用稠密算力卡对稀疏大显存主机进行加速,主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
Tools, ComfyUI workflows and benchmark configs from a 4x RTX 3090 local-inference rig
run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and windows, one line to install
Windows player that runs video, photos or GIFs through NVIDIA's DLSS 5 neural renderer, then compares the result with the original frame by frame. Local files or YouTube. Needs an RTX card and driver 610.47+. Community project, unsigned runtime. C++20, D3D12.
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
Enable NVIDIA Smooth Motion on RTX 30-series / Ampere SM86 - one-click Windows utility, no per-game DLSS-FG integration required.
Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.