wici-ai / inferssd

git clone https://github.com/wici-ai/inferssd.git v0.1.1 · 1d ago

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.