rulith-dev / rulith-inference

git clone https://github.com/rulith-dev/rulith-inference.git

Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.