BinceQu / RoboHarness

## A Simple Harness Outperforms VLA and World Action Models
**Project Page · Quick Start · Documentation · 中文**
 RoboHarness overview: visual keypoints are tracked persistently, converted into geometric constraints, executed, and verified in a closed loop. View original PDF 
## Overview
Without training models, RoboHarness combines visual clicks, keypoint tracking,
and geometric constraints to provide an embodied control panel for an LLM agent.
The LLM marks points of interest on a 2D image and continuously receives their
positions via optical-flow tracking and depth back-projection; with this
information, it composes accurate primitive actions to perform complex
manipulation.
The reported BEHAVIOR results use Claude Code 2.1.259 with
`Qwen3.8-Flash-Next-FP8` and the R1 Pro robot. A Codex harness is also available.
## What is included
| Component | Purpose |
| --- | --- |
| `BEHAVIOR/` | BEHAVIOR simulator and evaluator, pinned to v3.9.1 |
| `interface/` | RGB-D observations, robot control and interactive interface |
| `harness/claude_code/` | Claude Code harness |
| `harness/codex/` | Codex harness |
| `prompt/` | Task-specific instructions for household manipulation |
| `tasks/` | Task definitions, evaluation instances and step budgets |
| `reference_results/` | Reference evaluator scores (hosted on Hugging Face; `setup.sh` fetches them) |
| `roboharness/` | Task runner and evaluation pipeline |
| `validation_results/` | Evaluation reports and scores (hosted on Hugging Face) |
## Installation
Use Linux x86-64, an NVIDIA RTX GPU, Python 3.11 and the CUDA 12.4 toolkit.
The validation host has 48 GB VRAM per GPU; this is the tested capacity, not a
measured minimum. You also need BEHAVIOR scene data, local cache space and a
separately configured model endpoint.
 [code] 
Videos, reference results and validation reports are hosted in the
RoboHarness-assets
dataset, pinned by `assets.lock.json`, so cloning stays small. `setup.sh`
fetches the reference results the runner needs; `python3 scripts/fetch_assets.py --group all`
fetches everything and verifies each SHA-256. The paper and project page correspond to
the `v1.0-paper` release.
Follow the installation guide to obtain the licensed datasets,
install Claude Code 2.1.259, and configure the model endpoint and authentication.
Edit `configs/local.json` for this machine's data and Python paths.
## Run a task
The model server and the evaluation runner are separate processes. Keep your
model server running throughout evaluation. `setup.sh` installs the simulation
and agent environments; it does not download LLM weights or start a model server.
**1. Connect to your model service.**
The default Claude Code harness uses an **Anthropic-compatible `/v1/messages`
endpoint with image input, tool calls and streaming**. The paper configuration
uses `Qwen3.8-Flash-Next-FP8`. If your server only provides OpenAI Chat
Completions, follow the bridge setup
first. Codex uses a different Responses configuration.
On the **e