rzc156 / dreamforge

# dreamforge
**Turn world-model imagination into corrective training data for robot policies.**
A single-GPU, end-to-end research pipeline with a **falsifiable** core:
every imagined action is replayed in the real simulator before anything is
claimed about it.
English · 中文 · Protocol · Results · Dashboard
---
## Why this exists
A previous controlled study found that **test-time action selection** — sampling
several candidate chunks from a flow-matching VLA and picking the one a world
model likes best — has **near-zero headroom** on a strong policy, and that
offline world-model scores do not predict decision quality.
That leaves the obvious question:
> If imagination cannot **select**, can it **teach**?
`dreamforge` answers it end to end: mine the policy's own failure states, use a
latent world model to search *corrective* action chunks (CEM), **verify each
correction by replaying the exact simulator state**, and only then decide
whether it is worth training on.
## Headline results
LIBERO-Spatial · SmolVLA-450M · 150 mined episodes → **360 failure states** ·
**1080+ paired simulator replays** · single RTX 5080. Full log:
`docs/results.md`.
| experiment | result |
|---|---|
| world-model pre-flight (`diagnose`) | unregularized WM action response **0.0003** vs action-sensitive **0.0163** (~50×) |
| imagined progress (CEM) | 0.480 → 0.500 (+0.020) |
| **real value of corrections**, 360 paired states | none **9.4%** · imagined **13.6%** · random **14.7%** |
| **imagined vs none** | **+4.2pp, exact McNemar p = 0.032** |
| **imagined vs equal-budget random** | **−1.1pp, p = 0.61** — the model's direction buys nothing |
| training on corrections (`ft_all`) | 57.0% vs base 61.0% vs replay-only control 58.0% (**p = 1.0**) |
| corrections concentrated where the metric looked best (`ft_early`) | **45.0%** — 13pp *worse* than the control (p = 0.041) |
 What that means (click) 
World-model imagination changes outcomes, but **only as much as a random
perturbation of the same magnitude**, and training on it does not beat a
replay-only control. Worse, feeding corrections where the decision-point metric
says they help most *hurts*: those states are not reliably failures (the policy
succeeds from the same initial state most of the time), so imitating a
perturbation there overwrites correct behaviour.
The practical reading, and the reason the tooling is the point of this repo:
**before paying for a world model in a control loop, (1) measure its action
response, (2) benchmark its output against an equal-budget random perturbation,
(3) never assume a decision-point metric is a training signal.**
## Interactive dashboard
Every run renders a **self-contained HTML dashboard** (no CDN, no JavaScript)
from its artifacts — handy for a report, a PR, or GitHub Pages. It ships with a
**CSS-only English / 中文 switch** (hidden radio inputs + sibling selectors, so
the page stays script-free).
 [code] 
The committed copy lives at `results/dashboard.html`
(and the Chinese view is at