ggml-org / whisper.cpp
Port of OpenAI's Whisper model in C/C++
ggml-org / whisper.cpp
openvinotoolkit / openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
alibaba / MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
qpdf / qpdf
qpdf: A content-preserving PDF document transformer
OpenNMT / CTranslate2
Fast inference engine for Transformer models
2toinf / X-VLA
[ICLR 2026] The offical Implementation of "Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model"
brontoguana / krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
Carloscodix / qapla
A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C.
sauravsingla / MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.
ahmedbarakat207 / espllm
a Transformer Based Model for ESP32 (520KB Memory)