Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages
An Open Source Machine Learning Framework for Everyone
Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++