InternLM / Intern-Decision

Intern-Decision 
 🎮 Demo 
   ·  
 🤗 Model Weights 
 简体中文 ·
 Data interface ·
 Reproduce evaluations 
Intern-Decision turns a state, optional images, and multiple typed questions into
decisions with probabilities. It fine-tunes the language backbone of Qwen3.5
while freezing the vision tower and projector. Supported question types are
`choice`, `score`, and `noul` (yes/no).
This release contains training, two inference backends, benchmark scoring,
temperature calibration, and a browser demo. It also includes the **96-case
distribution calibration benchmark**,
its exact references, deterministic generator, and offline scorer, plus the
seven accuracy test suites and checkpoint-specific temperature presets. Training
data, private calibration/validation records, images, preparation pipelines, and
model weights are not included. The runtime schema, tokenization, and collation
support your own records. Demo examples are synthetic interface illustrations.
 🎬 Demonstrations 
 Click any preview to watch the full video. 
 🍄 Mario 
 🖱️ Mouse Moving 
 🛡️ Doom · Defend 
 💚 Doom · Health Gathering 
 🌐 Browser Use 
## Method
The objective is `autoregressive masked language modeling`. The assistant input contains a complete
JSON skeleton with one ` ` token per field. Ground-truth answer symbols
appear only in labels, never in the input. Training uses ordinary causal
next-token alignment: the logit immediately **before** a marker predicts that
field's answer. All fields share one causal forward pass.
Training cross-entropy uses the full vocabulary. Inference restricts logits to
the legal single-token answer symbols, applies softmax, and maps them back to
the original labels. Question and option ordering must be preserved.
## Results
Intern-Decision-4B reaches **90.02% average accuracy** across seven suites (Jev: 88.74%),
**44.16 ms mean local request latency** on an RTX 4090, and **0.550 Brier / 0.089 ECE**
on the 96-case calibration pilot (Jev: 0.595 / 0.130). The protocols and scope differ:
see accuracy, latency, and calibration.
### Benchmark accuracy
Accuracy columns are percentages; Average is the unweighted mean of the seven
displayed suites. The seven suites contain 10,751 rows and 12,351 decisions;
per-suite denominators are listed in Evaluation.
Brier and ECE are **Jevbench-Hard** metrics on 111 items. This table uses
**max(P)** confidence and 10 equal-width bins for ECE, displayed as a fraction;
Brier is the raw multiclass sum. Intern-Decision probabilities use the fitted
temperatures below, with unchanged argmax predictions. Baselines use their
reported/default probabilities; no additional temperature was fitted here.
| Model | Easy | Original | Hard | Typed Decision | ToolACE | AG News | WildJailBreak | Average | Brier ↓ | ECE ↓ | T |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| Jev | 100.00 | 98.61 | 72.07 | 73.35 | 91.29 | 89.57 | 96.29 | 88.74 | 0.358420 | 0.094685 | — |
| Laya | 95.83 | 72.22 | 28.83 | 35.95 | 63.87 | 92.84 | 14.8