lemonade-sdk / lemonade
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
lemonade-sdk / lemonade
mozilla-ai / llamafile
Distribute and run LLMs with a single file.
heterodoxin / AbliteratedOutTheBox
llama.cpp fork that abliterates at inference
alibaba / MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
microsoft / BitNet
Official inference framework for 1-bit LLMs
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
ROCm / FastFlowLM
Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.
OpenMOSS / MOSS-VL
An open-weight 11B model series for long-form and real-time video understanding
hyperfield / ai-file-sorter
Cross-platform desktop application for content-aware file organization and renaming. Supports local and remote LLMs, preview-based workflows, and fully user-controlled changes.
epoupon / lms
Lightweight Music Server. Access your self-hosted music using a web interface.
tsaipifong / whirl-llm
WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.
b4rtaz / distributed-llama
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
lean-dojo / LeanCopilot
LLMs as Copilots for Theorem Proving in Lean
epsilla-cloud / vectordb
A high performance Vector Database leveraging parallel graph computing
invergent-ai / surogate
Train and serve LLMs at extreme speed and massive throughput.
amethyst8118 / MetalCyan
Metal acceleration project for the ASRock BC-250 on macOS. Based on ChefKiss' NootedRed project. LLMs are used in this project for debugging.