MaximeRivest / tiny-classifiers

# tiny-classifiers
**Train your own 17-million-parameter text classifier in about a minute, with labels made by a big AI model,
and know when it beats an API.**
A 17M model (Ettin), fine-tuned in 42 seconds on one gaming GPU, sorts bank messages into 77 categories about
as well as Claude Opus (91.5% vs 92%), for about $0 instead of $8,120 per million messages, at 5 ms a message.
It also trains on a laptop CPU (11 min) and on a phone's GPU (21 min).
This repository has everything behind those numbers: the recipe, a step-by-step tutorial, the measured results
on ten public tasks, the phone trainer, and the source of the explainer video.
## What we found
- **The model is the easy part; the labels are the whole game.** More labelled examples keep helping, for every
 model size we tried (learning curves).
- **No labels? Borrow a teacher.** A big model (Kimi K3) labels your messages: 2,000 in about 20 seconds for
 about $1. The student trained on them lands **a few points under its teacher**; human labels break that ceiling.
- **Ten tasks, one recipe.** On human labels, Ettin 17M beats Jev (a zero-shot classification API) on 6 of 10
 tasks. On Kimi's labels, it ties Jev on 2 and is within 5 points on 5 more; it loses where the teacher itself
 is weaker (table).
- **When it pays:** against Jev's price, fine-tuning pays for itself after roughly **50,000 to 500,000 messages**
 (per task). Below that, or before you know you need scale, use an API.
- **Evaluate first.** To trust any model in production, you need a few hundred messages with checked answers.
 Those are also most of what you need to train your own.
## Start here
**The tutorial** — `tutorial/distill-ag-news.md`: a runnable notebook, top to
bottom in about 10 minutes and $1.50 of API credit. Tidy ten datasets, label AG News with two teachers
(Kimi K3 and Qwen3.8-27B), fine-tune Ettin-17M on each set of labels and on the human labels, compare, and use
the model. Our run: human labels 89.2%, Kimi's 85.1%, Qwen's 81.8%; 6 ms per message on a CPU.
**The recipe**, four scripts (uv installs everything on first run):
 [code] 
 [code] 
| script | what it does |
|---|---|
| `recipe/prepare.py` | Downloads banking77, CLINC150, MASSIVE (EN, FR), TREC, AG News, SST-5, Financial PhraseBank, LEDGAR and tweet hate speech from the Hugging Face Hub; builds `task.json` (one instruction sentence + the labels), 2,000 training messages, up to 150 per category (`train_big`), and the test set. |
| `recipe/label.py` | Labels messages with Kimi K3 across several providers at once, as fast as each allows (adaptive concurrency, retries, hedging of slow requests, a budget per provider). Resumable. Reports time, cost and, on the test set, the teacher's accuracy. |
| `recipe/train.py` | Fine-tunes all weights of Ettin-17M (or any encoder, `--model`) on human or teacher labels, on a GPU or a CPU; prints test accuracy; saves a standard Hugging Face model. |
| `recipe/breakeven.py` | After how many messages the upfront cost (la