使用稠密算力卡对稀疏大显存主机进行加速,主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and windows, one line to install
RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.