eagle-hq / EAGLE-4

EAGLE-4 
 One Flexible Drafter for All Batch Sizes and Hardware 
 Paper (EAGLE-4) ·
 EAGLE-4 draft for Qwen3.8-27B ·
 Documentation 
 Figure 1(a) of the EAGLE-4 paper: Qwen3.8-27B on the SPEED-Bench qualitative split, 4× NVIDIA B300, one request, temperature 0.
 The numbers this release measured with EAGLE-4 are in the support matrix .
## EAGLE-4
EAGLE-4 (Extrapolation Algorithm for Greater Language-model Efficiency) is the latest generation of the EAGLE family
for accelerating large language model inference. One drafter adapts its speculative decoding strategy to the batch
size, the hardware and the workload, and keeps the target model's output distribution.
The EAGLE-4 draft uses a sliding-window draft layer, a low-rank candidate head and an append-only draft cache. Dense
and latent-MoE draft structures are implemented. Qwen3.8-27B with its
EAGLE-4 draft is the measured target pair.
## Engines
This repository is `eagle-harness`, a pip-installable package (`eagle_harness`) that serves EAGLE-4 drafts on
SGLang, vLLM and TokenSpeed through engine plugins. EAGLE-4 sizes the speculative tree from measured draft and
verification costs for the batch size, hardware and workload it is running on. It registers through hooks the engine
exposes and does not patch the engine at runtime.
The patched engines ship as pinned wheels and container images, not as source trees in this repository:
| Engine | Hardware | Wheel index | Image | Source |
| --- | --- | --- | --- | --- |
| SGLang | NVIDIA B300, CUDA 13.0 | cu130 | `ghcr.io/eagle-hq/eagle-4-sglang:0.1.0` | eagle-hq/sglang |
| SGLang | AMD MI355X, ROCm 7.2 | rocm | `ghcr.io/eagle-hq/eagle-4-sglang-rocm:0.1.0` | eagle-hq/sglang |
| vLLM | NVIDIA B300, CUDA 13.0 | cu130 | `ghcr.io/eagle-hq/eagle-4-vllm:0.1.0` | eagle-hq/vllm |
| TokenSpeed, experimental | NVIDIA CUDA only | cu130 | `ghcr.io/eagle-hq/eagle-4-tokenspeed:0.1.0` | eagle-hq/tokenspeed |
Stock engine packages do not carry the hooks. Release settings is the one place that holds
every version pin, index URL and image tag. The wheels are also attached to the
v0.1.0 GitHub release.
## Measured results
The support matrix reports every measurement with its conditions, separately for
loading, greedy correctness, request features and throughput. Throughput is generated tokens per second of
workload time, higher is better. The rows below are the shapes the quickstarts ship. Each one is the mean of
two server starts on the SPEED-Bench client at temperature 0.
| Engine | GPUs | Client concurrency | No-spec tokens/s | EAGLE-4 tokens/s | Speedup |
| --- | --- | ---: | ---: | ---: | ---: |
| SGLang | 4× B300, TP4 | 8 | 1479.4 | 2820.6 | 1.91× |
| SGLang | 4× B300, TP4 | 64 | 3925.3 | 5281.3 | 1.35× |
| SGLang | 1× MI355X, TP1 | 8 | 552.7 | 1145.9 | 2.07× |
| SGLang | 1× MI355X, TP1 | 32 | 1269.2 | 1980.8 | 1.56× |
| vLLM | 4× B300, TP4 | 1 | 241.6 | 722.8 | 2.99× |
| vLLM | 4× B300, TP4 | 8 | 1699.9 | 2883.9 | 1.70× |
Each speedup divides EAGLE-4 throughput by th