Zefan-Cai / Open-Jev

# Open-Jev
**Open-Jev-27B-v1.1 is here. Give your app a decision, with probabilities.**
Supply a context, questions and candidates; get typed probabilities directly,
without autoregressive answer generation or parsing generated JSON.
### Same preview. Different target.
**Watch the 32-second demo and try the interactive replay →**
On X: new 27B v1.1 video ·
benchmark chart ·
poster and release links.
Two confirmation-gate cases. A misleading preview. Seven possible decisions.
Inspect how Open-Jev-27B-v1.1 ranks them, then compare the saved probabilities
when the actual target changes. The video replays real model outputs from
synthetic held-out cases; it is edited playback, with no browser actions executed.
Candidate order also differs between the original cases, so this is not a
controlled single-variable experiment.
**Build with it:** Model ·
Public dataset ·
Demo video ·
Poster ·
Replay evidence
**Project website:** zefan-cai.github.io/open-jev — demos, measured results and project background.
**Hugging Face:** Collection ·
2B model ·
9B model ·
27B v1.1 model ·
v1.1 dataset ·
Original dataset
**Try it online:** Text & CSV workbench · Open in Hugging Face
**No-code workbench:** User guide · Workbench source · CPU deployment. Paste messages or import a CSV, define your categories, and download the original table with suggested labels and candidate probabilities. The hosted workbench uses the released Open-Jev-2B model on CPU. Start with a few rows: this trial is slower than the GPU deployment measured below. Its optional walkthrough is explicitly marked as an illustrative example.
**Evaluation versions:** Public, independent and pending results.
The historical 231-public result remains separate from JevBench v1.4 and the
v1.5.4 independent snapshot. On v1.5.4, 9B is #46 / 24.356 and 2B #65 / 9.073;
its chance-corrected competence is not raw accuracy. Fresh independent 27B v1.1
results remain pending. Prepare the pinned evaluator handoff.
**Deployment:** Trained-model quickstart and GPU / CPU / MPS matrix.
**Measured acceleration:** attributed open-jev-fast integration and an original
Triton final-row RMSNorm/head kernel are available as experimental opt-ins. On
H200, the isolated operator showed 2.63–2.75× warm P50 ratios. Released-2B
whole-model ratios were only 1.006–1.020×, without established statistical
significance; 336 reference comparisons had no decision or threshold flips.
Both backends peaked at 6.917 GiB on the recorded workloads. A released-27B H100
smoke executed all three backends: fast-CUDA failed the unchanged probability
tolerance (0.00593 versus 0.0001). Full Torch/Triton-tail validation then
passed 336 comparisons across eight workloads, with maximum probability error
1.89e-6 and no decision or threshold flips. Whole-model P50 differences remained
small (1.004–1.020×); statistical significance was not established.
Results and implementation.
**Natural support pilot:** the released 2B scored 166/256 (64.8%) versus BM25's
211/2