Edge0-AI / Edge0
git clone https://github.com/Edge0-AI/Edge0.git
Streaming MoE inference framework that runs 35B sparse models on consumer devices by offloading expert weights to SSD with predicted routing.
Why you might care
Targets running large mixture-of-experts models with peak memory bounded by the active expert set rather than total parameters, on Mac, iOS, Android, and Windows. A trained prerouter prefetches experts a step ahead, reported up to +59% decode throughput; Recover-LoRA adapters recover most 4-bit quantization loss. Platform engines are at varying maturity, with the unified framework slated for Q4 2026.
Star history (as recorded here)
Links
- GitHub: github.com/Edge0-AI/Edge0
- Releases · Issues · Hacker News mentions
README
Loading README…
Blurb generated by claude-haiku-5-5 today. Star deltas marked ~ are extrapolated from partial history.