jayleaton / deepseek-v41-tensorfold-spark

Follow me on X for more updates: https://x.com/jayleaton
Support me here: https://buymeacoffee.com/jayleaton
# DeepSeek-V4.1-Flash-1017 on TensorFold, 2x NVIDIA DGX Spark
A recipe for serving DeepSeek-V4.1-Flash-1017 on two NVIDIA DGX Sparks behind an OpenAI-compatible API. It ships
our EXL3 pack (**flash17**) and the TensorFold Zig engine that serves it, with the exact serving profile we measured.
| | |
| --- | --- |
| Model | `deepseek-ai/DeepSeek-V4.1-Flash-1017` |
| Pack | **EXL3 2.8 bpw, 203.8 GB**: `jayleaton/DeepSeek-V4.1-Flash-1017-EXL3-2.8bpw` |
| Hardware | 2x DGX Spark (GB10, 128 GB each), tensor parallel 2 over the 200 Gb/s CX7 link |
| Engine | TensorFold Zig, CUDA port `9131e10a` (source in `engine/zig`) |
| Serving | 1M context, four slots, vision (native image input), tool calling, structured output |
flash17 is serving as the production default on our two Sparks (model ID `DeepSeek-V4.1-Flash-TF`).
> **Work in progress.** Measured on one pair of Sparks. Read what is not solved before
> relying on it.
## Results
**Tasks.** 356 matched items (pass@1, greedy, thinking on, effort high), against Mia-AiLab's 2.9 bpw pack and
our previous q28-v2 pack:
| Suite | flash17 | Mia 2.9 bpw | q28-v2 |
| --- | ---: | ---: | ---: |
| HumanEval+ | 34 / 40 | 34 / 40 | 33 / 40 |
| MBPP+ | 50 / 60 | 49 / 60 | 50 / 60 |
| MMLU-Pro STEM | 93 / 100 | 91 / 100 | 90 / 100 |
| MATH-500 | 36 / 40 | 36 / 40 | 36 / 40 |
| SciCode (first step) | 6 / 10 | 6 / 10 | 5 / 10 |
| LiveCodeBench | 16 / 20 | 13 / 20 | 13 / 20 |
| BFCL | 39 / 40 | 39 / 40 | 39 / 40 |
| Tool chains | 6 / 6 | 6 / 6 | 4 / 6 |
| MGSM | 40 / 40 | 40 / 40 | 40 / 40 |
Small samples: every paired 95 % confidence interval includes zero, so these show no statistically significant
difference. Intervals, method and limits are in FLASH-1017.
**Retention of the 1017 release, as served** (top-1 agreement with the release, with the serving expert pruning):
code **95.17 %**, science **94.93 %**, tools **87.64 %**. The full table, with KL and the other packs:
FLASH-1017.
**Speed** at the recommended profile (2.1M pool, 2100 MHz cap), single stream: short decode **87.9 tok/s**; decode
after a 32K / 131K prompt 64.1 / 64.9 tok/s; prompt tokens over time to first token 2,349 / 2,198 tok/s (TTFT includes
HTTP and admission). q28-v2's earlier measurements, under a different clock and its 3.0M pool, were 85.75, 66.7 / 64.5
and 2,630 / 2,460; not a matched A/B. Details: FLASH-1017.
**Images and agents** at the recommended profile: VQA 20 / 20 plus 3 / 3 `image_url` probes; a clean OpenCode
session passed (7 valid tool calls, normal stop).
## Quick start
The full guide is ZIG-SERVE. Build the image and the per-rank assets as it describes, then:
 [code] 
- Fill in the node paths and link settings from
 `config/prod-zig-flash17.env.example` and keep its engine settings. It needs
 the **2.1M-token pool** (`TF_DSV41_POOL_TOKENS=2100000`). With q28-v2's 3.0M pool this pack goes under the 4 GiB
 memory floor during 4K