robertkarl / UnbiasedStrata

## Credits and license
The engines are PolyStrata's, by VecSzn and contributors, built on Strata by
Niko1221. The front end is built from llama.cpp / ggml's server, common and
mtmd code. Qwen3.8-Flash-Next and Qwen3.6-35B-A3B are by the Qwen team; the IQ3_XXS GSQ-RCO file is
ISTA-DASLab's, the Qwen3.6 files are
Unsloth's. Upstream's full credits: docs/HOW_IT_WORKS.md.
MIT License (LICENSE); some parts and every model have their own licenses
(which ones).
# UnbiasedStrata 2 - note from a human.
Issues with Strata / PolyStrata:
* piping instructions from markdown into agent == bad idea
* replacing components at build-time or runtime isn't great
* desired: clone; submodule init; make; situation
UnbiasedStrata aims to be the suckless (dwm) variant of Strata.
It's been vetted in a few ways:
* agents went through and verified kernels to the difference between CPU/GPU within llama.cpp
* full a run of Qwen 3.8 Flash Next over Bob-bench wide test set with a high score.
This wouldn't exist without llama.cpp.
without Niko's Strata.
Without PolyStrata either.
Use at your own risk. Works on my machine.
# AI generate nonsense from here on out
A fork of PolyStrata v0.1.45-poly.1 that keeps its two inference engines
unchanged and replaces everything around them: no installer, no downloads at build or start, no Python between a
request and a token. One `make build` on a Linux box with an NVIDIA card produces the engines and a C++ front end,
`unbiased-strata`, that serves an OpenAI-style chat API from llama.cpp's own server code. The target is one NVIDIA GPU,
Pascal or newer, CUDA 12.x, Linux x86-64, on a machine that may have no network at all. What was changed and why is in
UPSTREAM.md; the plan this follows is PLAN.md.
 [code] 
## Run
 [code] 
`-m` names the GGUF the vocabulary and chat template are read from; no tensors are loaded from it by the front end.
`--engine` takes a path, or a bare name looked up beside the program. Everything before `--` is a `llama-server`
option (`--host`, `--port`, `--api-key`, `--alias`, `--temp`, `--top-k`, `--top-p`, `--min-p`,
`--reasoning-format`, `--chat-template-kwargs`, `--sse-ping-interval`, ...); everything after `--` goes to the
engine, which is started as a child process with `--serve` added and spoken to over its line protocol
(docs/BACKENDS.md). Where the model ends up (GPU cache, pinned RAM, SSD) and the three expert
modes: docs/MEMORY_LAYOUT.md. The server answers `/v1/chat/completions` (streaming and not), `/v1/models`,
`/health`, `/props`, `/tokenize`, `/detokenize`, as llama.cpp's server does, one request at a time.
 [code] 
With pictures, add `--mmproj /models/mmproj-F16.gguf` (the projector from the same `-MTP` repository) and
`--threads N` for the encoder before the `--`. The front end checks the projector's width against the model's at
start, encodes each picture with llama.cpp's mtmd and hands the rows to the engine over upstream's `GENI`
protocol. A picture is at most 300 rows by default (upstream's instal