rockmatrix-ai / EdgeRock

EdgeRock 
 Strong Reasoning at Constant Memory ROCK MATRIX · rockmatrix.ai 
 English · 简体中文 
**EdgeRock brings strong reasoning to a fixed history budget.** EdgeRock-4B, our first open-source release, supports text and image reasoning. Its constant-size history leaves more GPU memory available for sustained reasoning and concurrent serving. More model sizes are in preparation.
- **Strong mathematical reasoning:** **82.9%** mean AIME pass@1, versus 18.8% for the strongest evaluated bounded-state baseline.
- **O(1) history memory:** fixed **72 MiB** attention KV per sequence—**99.1% less** than Qwen3.5-4B.
- **Higher serving throughput:** **6.42×** Qwen3.5-4B's output throughput at each model's best measured batch.
Quick start · Key results · Models · Documentation
*AIME is mean pass@1 across 2024–2026; KV is compared at 256K context. Serving throughput uses 32K input and 32K output on one Hopper 80GB GPU at each model's best measured batch. It counts output tokens only, with elapsed time including prefill.*
## Quick start
| Model | Input modalities | Model weights |
| --- | --- | --- |
| **EdgeRock-4B** | Text, images | Hugging Face · ModelScope |
 [code] 
For optimized inference and serving, see the inference guide.
## Model directory
Choose a package by runtime, or browse the public EdgeRock model collection. Each model card contains its installation instructions, license, and validation scope.
### Original model and compact PyTorch
| Model repository | Format / runtime |
| --- | --- |
| EdgeRock-4B (original) | BF16 · text and images |
| EdgeRock-4B-NF4 | NF4 · included compact loader |
| EdgeRock-4B-INT8 | INT8 · included compact loader |
| EdgeRock-4B-FP8 | FP8 · included compact loader |
| EdgeRock-4B-AWQ | AWQ 4-bit · included packed INT4 GPU runtime |
| EdgeRock-4B-GPTQ | GPTQ 4-bit · included packed INT4 GPU runtime |
| EdgeRock-4B-NVFP4 | NVFP4 W4A16 · BF16 reference loading |
The compact packages validate text inference on Hopper; see each card for its conversion checks. Original-model benchmark scores do not establish quantized-package accuracy or throughput.
### MLX
| Model repository | Format / runtime |
| --- | --- |
| EdgeRock-4B-MLX-8bit | 8-bit · group 64 |
| EdgeRock-4B-MLX-6bit-g64 | 6-bit · group 64 |
| EdgeRock-4B-MLX-4bit-g32 | 4-bit · group 32 |
| EdgeRock-4B-MLX-4bit-mixed | Mixed 4-bit |
These packages use the included EdgeRock MLX runtime. Functional checks were run on Linux CUDA; Apple Metal has not been tested.
### GGUF and ONNX
| Model repository | Format / runtime |
| --- | --- |
| EdgeRock-4B-GGUF | 7 profiles · custom EdgeRock backend |
| EdgeRock-4B-ONNX-Q4-g32 | 4-bit · ONNX Runtime CUDA |
| EdgeRock-4B-ONNX-Q4-CPU-g32 | 4-bit · ONNX Runtime CPU |
| EdgeRock-4B-ONNX-Q8 | 8-bit · ONNX Runtime CUDA |
GGUF requires the custom EdgeRock RMA backend; stock llama.cpp/Ollama is not supported. Bundled native binaries target Linux x86-64 Hopper. ONNX packages require their supplied stateful runner; mobile ONNX Runtime has