rkinas / basal

# basal 2.3
Inference engine and models for **basal-2.0** and **basal-omni-1.0**: small, fast, calibrated *typed-decision* models
for Polish and English. The basal-omni models also take **images and audio** in a request (experimental).
**What it is.** basal is inspired by *System 1* (fast, intuitive) decision models such as Jev: instead of a chatbot
that writes an answer, the model reads a **state** (a message, a document, a case file, retrieved passages, an agent
trace, a web page as JSON) and answers a **typed question** about it by returning a **probability for each allowed
answer**, in one forward pass, without generating text. The answer can never fall outside the options you gave, and
the probability says how sure the model is.
**What it is for: a dynamic classifier.** You describe the classes *in the request*, in plain language, per call,
instead of training a classifier for them: routing tickets and documents (up to 255 options), rule and policy
checks, questions about long documents, RAG decisions, scoring on ordered scales, the next step of an agent, judging
answers, and triage with a confidence threshold that sends uncertain cases to a person.
> The name comes from the *basal ganglia*, the part of the brain that selects one action among competing options.
## Contents
- What is new in basal 2.3 · What is new in basal 2.2 ·
 What is new in basal 2.1 · What is new in basal-2.0
- Models · Results
- Quick start · API
- Images and audio: basal-omni · MCP for agents
- Documentation: install paths, serving, API reference, results, formats, hardware
- Limitations · Previous releases · Citation ·
 Contributors · License
## What is new in basal 2.3
- **basal-omni on the fast engine** (`--mode fast`, text): the CUDA graph engine now runs basal-omni (Gemma 4). A
 request's state is computed once for every question, mini's last 18 layers run only where answers are read, and the
 layers are compiled with FlashAttention 3 and FlexAttention. Decisions per second at 16 requests in flight, against
 2.2's `--mode eager` on the same GPU:
 | model | GPU | one question | five questions about a long document |
 |---|---|---|---|
 | basal-omni-1.0-mini | H100 | 10.0 -> 61.2 (6.1x) | 1.6 -> 21.4 (13.7x) |
 | basal-omni-1.1-max | H100 | 4.8 -> 16.2 (3.4x) | 0.72 -> 10.6 (14.7x) |
 | basal-omni-1.0-ultra | H100 | 2.0 -> 6.1 (3.0x) | 0.28 -> 4.8 (17x) |
 | basal-omni-1.0-mini | RTX PRO 6000 Blackwell, `--gemma-attn flex` | 5.6 -> 23.2 (4.1x) | 0.85 -> 18.4 (21.6x) |
 The same answers within bf16 rounding: on 10,606 development decisions the top answer matches 2.2 on 99.85-99.99% of
 those where the model is sure, and with the shipped calibration files the share automated at 1% error moves by at
 most 0.1 points. Without `--mode`, basal-omni now runs `fast` on Hopper GPUs with FlashAttention 3 and `eager`
 elsewhere (in 2.2 it stopped at start-up). docs/OMNI.md
- **A faster `--mode eager` for basal-omni** (images, audio, `--deterministic`): the state, and the image