lxe / dsx

# dsx
dsx runs **DeepSeek-V4.1-Flash-1017** (552B MoE + 196B Engram) at full routing on a desktop with two RTX 3090s,
128 GB of RAM and an NVMe drive. It is a Rust + CUDA engine built for this one model, and it runs its own 2-bit quant
(lxe/DeepSeek-V4.1-Flash-1017-dsx-2bit).
- ~29 tok/s decode, 300-470 tok/s prefill on 2x RTX 3090
- Exact routing: no expert pruning, no router bias
- OpenAI and Anthropic compatible server with tool calls, streaming, image input and a built-in web UI
- Quality vs the FP8 model (Lucebox eval): KL 0.373, top-1 74.4%
| | 2x RTX 3090, 128 GB DDR4 | RTX 5090, 128 GB DDR5 |
|---|---:|---:|
| Decode, prose | 28.9 tok/s | 12.0 tok/s |
| Decode, 3K-token agent prompt | 28.7 tok/s | 15.2 tok/s |
| Prefill, 3K / 8K tokens | 301 / 470 tok/s | 93 / 172 tok/s |
Both machines run the 2-bit quant, but the columns were measured with different builds of it. The RTX 3090 numbers
come from build v4, which has the same expert layout as the published build (v4b); the RTX 5090 numbers come from the
older build v2. The single RTX 5090 cannot hold every expert in VRAM and RAM, so about 40 GB of experts are read from
disk during decode, and that is what limits its speed.
## Quick start
 [code] 
Open `http://localhost:8090/` for the web UI, or use any OpenAI client with base URL `http://localhost:8090/v1`:
 [code] 
## Requirements
| | |
|---|---|
| GPU | NVIDIA with 24 GB or more and CUDA 13. It has run on the RTX 3090 (sm_86), RTX 4090 (sm_89) and RTX 5090 (sm_120); an A100 (sm_80) build compiles with `CUDA_ARCHS=80` but has not been run |
| RAM | 128 GB |
| CPU | x86 with AVX2 (AVX-VNNI or AVX-512 VNNI preferred) |
| Disk | ~350 GB: the quant (148 GB) + the Engram shards (203 GB) |
The quant's routed experts take about 140 GB. dsx keeps as many of them as fit in VRAM and in a RAM arena, and reads
the rest from the GGUF file on disk, which slows decoding. Two 24 GB GPUs with 128 GB of RAM hold all of the experts;
one 32 GB GPU with 128 GB of RAM leaves about 40 GB of them on disk.
## Parameters
`dsx-server` needs `--model`, `--engram` and `--tokenizer`; every other option has a default.
**Model files**
| | |
|---|---|
| `--model` | the quant's first GGUF shard (the others load from beside it) |
| `--engram` | directory with the official checkpoint's Engram shards (`model-00047/00048-of-00048.safetensors`) |
| `--tokenizer` | the official `tokenizer.json` |
| `--rank` | expert hotness order (`expert-rank.json` in the quant repo) |
| `--vision` / `--no-vision` | image input (default: `vision.safetensors` beside the model); `--no-vision` for text only |
**Server**
| | |
|---|---|
| `--host`, `--port` | listen address (default `127.0.0.1:8090`) |
| `--api-key` | require this key (`Authorization: Bearer ` or `x-api-key`; env `DSX_API_KEY`); put a TLS proxy in front of a public address |
| `--model-name` | model id reported by `/v1/models` (default `deepseek-v4.1-flash`) |
| `--ctx` | context window, prompt + generated tokens (default 32768) |
|