sergqwer / strata-nvfp4

# Strata NVFP4
**Qwen3.8-Flash-Next (125B hybrid MoE) with NVFP4 experts on one RTX 20, 30, 40 or 50 card (12 GB of VRAM or more)
and 64 GB of RAM or more, text and pictures.** A fork of Niko1221/Strata, on
upstream 0.2.0-beta (its engine, server and setup; not its app or installers). Setup installs, ready-made from Hugging Face, the original Qwen in NVFP4 (our GPTQ, or NVIDIA's
checkpoint converted), censored as Qwen ships it; on ours an opt-in switch turns the censorship off. Upstream's GGUF
models are still offered.
## Models
Accuracy against Qwen3.8-Flash-Next with every routed expert in Q8_0, speed on an RTX 5090 with 128 GB of RAM.
Setup downloads the models it names; each Hugging Face card has its measurements and how to run the
files by hand.
| Model | Same top token as Q8_0 | Decode, tokens/s | 32K prompt | RAM in use | PC for this speed | Get it |
| --- | ---: | ---: | ---: | ---: | ---: | --- |
| Q8_0: every expert Q8_0 (ours, `qwen-q8`) | 100% (the reference) | 47 | 10.3 s | 106 GiB | 128 GB | Maximilian228/Qwen3.8-Flash-Next-Q8_0-Strata |
| NVFP4 + Q8_0 mix (ours, `qwen-nvfp4-mix`) | 95.0% | 91 | 4.2 s | 105 GiB | 128 GB | Maximilian228/Qwen3.8-Flash-Next-NVFP4-Q8-Mix-Strata |
| Unsloth UD-Q4_K_XL (`unsloth`, `--model UD-Q4_K_XL`) | 94.1% | 169 | 4.9 s | 77 GiB | 96 GB | unsloth/Qwen3.8-Flash-Next-GGUF |
| NVFP4 GPTQ (ours, `qwen-nvfp4-gptq`, setup's default) | 93.6% | 177 | 3.9 s | 69 GiB | 96 GB | Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata |
| NVIDIA NVFP4, converted by us (`qwen-nvidia-nvfp4`) | 92.9% | 167 | 3.9 s | 69 GiB | 96 GB | Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata |
| Unsloth UD-Q3_K_XL (`unsloth`, `--model UD-Q3_K_XL`) | 90.7% | 208 | 3.8 s | 58 GiB | 96 GB | unsloth/Qwen3.8-Flash-Next-GGUF |
| ISTA-DASLab IQ3_S (`qwen`, `--model IQ3_S`) | 90.0% | 228 | 3.9 s | 52 GiB | 64 GB | ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF |
| ISTA-DASLab IQ3_XXS (`qwen`, `--model IQ3_XXS`, upstream's pick) | 87.3% | 239 | 3.9 s | 45 GiB | 64 GB | ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF |
| ISTA-DASLab IQ2_XS (`qwen`, `--model IQ2_XS`) | 85.0% | 265 | 4.2 s | 38 GiB | 48 GB | ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF |
| ISTA-DASLab Q2_0 (`qwen`, `--model Q2_0`) | 84.1% | 256 | 3.9 s | 37 GiB | 48 GB | ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF |
- **Accuracy:** how often the model's most likely next token is the Q8_0 model's. Measured on 5,153 tokens of code,
 docs, English and Ukrainian chats and agent transcripts, every model fed the same text. Every model ran with its
 experts on the CPU and the route tail skip off, so only the quantization differs. The Q8_0 model also has Q8_0
 attention, Qwen's BF16 router and Qwen's own FP8 PLE, so it is the original's precision everywhere but the 8-bit
 weights.
- **Speed:** RTX 5090 (32 GB), Ryzen 9 9950X3D, 128 GB of DDR5-5600, Windows 11, Strata NVFP4 0.1.42-nvfp4.1's engine with
 the arguments 0.1.42-nvfp4.2's setup writes for each model. Median of 3 runs: 4 greedy c