nokia-applied-research / AnyJev

**English** · 简体中文 · ⚡ Serve it · 📊 Results · 🏆 Leaderboard · 🧭 Roadmap · 📖 Levels
 Jiamu Zhang 1     Tianze Yang 1     Yucheng Shi 2     Evan Chen 1 
 Zixiang Nie 1     Kelly Wan 1     Liangjie Hong 1     Ninghao Liu 3     Liang Wu 1 
 1  Nokia, Sunnyvale, CA, USA      2  Tencent Hunyuan      3  The Hong Kong Polytechnic University 
 Qwen3-8B on a real BANKING77 item. Every number is a model output. 
> [!TIP]
> **🆕 Tacit: AnyJev's self-distilled models, 1.7B to 9B, are on
> Hugging Face.** One
> forward pass per decision. With `adaptive=True`, a capped share of low-confidence decisions goes to
> the model's own reasoning. They run on transformers, in-process vLLM or a stock `vllm serve`.
> `pip install "anyjev[hf]"`, then `Tacit.from_pretrained("morriszjm/Tacit-9B")`.
> Start here ↓
> [!NOTE]
> **🏆 No training, on a public leaderboard.** The training-free `Decider` (L0, two rotations) on the
> unchanged Gemma-4-31B has a Full score of **60.22** on the Decision Index 0.3,
> 5th of 149 entries on 2026-10-12. Details ↓
## ⚡ Serve it
**A Tacit model, in Python.**
 [code] 
`kind` is `"choice"` (default), `"yes_no"` or `"score"` (`options` are ordered levels, lowest
first, numbered from 1; `first_level=0` numbers them from 0). `decide_batch([...])` takes a list of such dicts.
**Send the hardest decisions to reasoning, with a cap.**
 [code] 
A decision whose top two options are less than `tau` apart in log-probability is answered again with
thinking on. After the reasoning, the answer is read as a label distribution, so it also has
probabilities and cannot fail to parse. At most `max_cot_share` of the last `cot_window` decisions
escalate, however hard the traffic gets. Every escalation is reported (`route == "cot"`, with the
first pass kept), and `tacit.stats` counts them.
**On vLLM.** A stock server; the client needs only the tokenizer.
 [code] 
 [code] 
`engine="vllm"` (`pip install "anyjev[vllm]"`) runs vLLM in the same process instead.
**An HTTP endpoint for many clients sharing one cap.**
 [code] 
`POST /v1/decide` takes one decision or `{"items": [...]}`; `GET /v1/stats` reports the escalation
share. The gateway listens on 127.0.0.1 and has no authentication of its own.
| argument | default | meaning |
|---|---|---|
| `adaptive` | `False` | send low-confidence decisions to the model's own reasoning |
| `tau` | `0.5` | low confidence: the log-probability gap between the top two options is below `tau` |
| `max_cot_share` | `0.2` | at most this share of the last `cot_window` decisions escalates; `None` removes the cap |
| `cot_window` | `1000` | how many recent decisions the cap counts; `None` counts every decision since loading |
| `cot_max_tokens` | `8192` | reasoning budget of one decision |
| `engine` | `"transformers"` | `"vllm"` runs vLLM in this process; `"server"` uses a running `vllm serve` at `base_url` |
**Any other model, training-free.** The `Decider` reads the same typed decisions from any open
causal LM, with no training and no labels