google-research / veriharness

Scaling Agentic Verification for Long-Horizon Tasks 
 Same model. Better evidence. 
Check what agent outputs disagree on, and what they all get wrong. Deliver a better artifact with a record of why. 
VeriHarness is a verification harness for long-horizon general agents. Given
a task and *N* independent rollouts of the same agent on it, VeriHarness uses
**the same model that generated the rollouts** to check them against the task
environment and to deliver one artifact that is better than a typical rollout,
together with a record of the evidence behind every change.
The verifier is not a stronger judge. Its advantage comes from two sources
only: the structure of the rollout pool, and evidence it actively acquires
from the environment (files, data, recomputable quantities).
## Overview and results
*Left:* the verifier is the generator's own model inside a harness. A
disagreement resolver tests the claims on which the *N* rollouts differ, and a
consensus challenger seeks evidence against the claims they share;
adjudication combines their findings to select and revise the artifact.
*Right:* with Gemini 3.5 Flash as both generator and verifier, VeriHarness
against a single rollout and the best prior
LLM-as-a-Verifier on the five benchmarks. Across the five benchmarks and two frontier models it achieves the
highest selection scores among the evaluated baselines, and evidence-backed
revision raises the average gain over a single rollout to 6.2 points with
Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. The full pool of about 26,000 rollouts behind
these numbers is released (see Data).
## Contents
- Overview and results
- How it works
- Repository layout
- Setup
- Usage
- Benchmarks and grading
- Data
- Isolation
- Native environments (optional)
- Operating notes
- Skills
- Citation
## How it works
Rollouts of one model either disagree on a claim or agree on it, and the two
cases call for different checks:
* **Disagreement → resolve.** The alternatives are already on the table. The
 *resolver* picks the check that best separates them, runs it against the
 environment, and eliminates the candidates the evidence contradicts.
* **Consensus → challenge.** Agreement is not evidence: rollouts share a
 model and therefore blind spots. The *challenger* looks for ways a shared
 value, a shared reading, or a shared omission could be wrong, and tests
 them.
Each task runs four model turns in three mutually isolated sessions:
 [code] 
Adjudication names a **base** rollout, an evidence-backed **revision plan**
(`work`), and the claims the evidence left **unresolved** (`open`). Selection,
revision and reconstruction are one knob: an empty plan returns the base
unchanged, a non-empty plan revises it, and `base: "none"` rebuilds the artifact
from the task inputs.
The program only materializes workspaces, routes files between sessions and
collects outputs. Which claims to examine, which checks to run and when to stop
are the model's decisions. The single programmat