apromisedland / VLCoT

# VLCoT
**Stage-Verifiable Latent Chain-of-Thought with Local Repair for Long-Horizon Robotic Manipulation**
Paper and project page · 中文说明 · Method and implementation choices · Data contract
This repository implements the paper's stage-aligned latent reasoning, observation-grounded verification, and dependency-aware suffix repair. It includes an OpenVLA-7B adapter, continuous action heads, offline annotation, LoRA training, LIBERO/RoboTwin environment adapters, evaluation accounting, ablations, and tests.
**Release status:** this is a paper-based research implementation with documented engineering additions. Original experiment code, trained VLCoT weights, stage annotations, and run-level records were not supplied. This release has not run training, inference, simulation, tests, or CI. Paper-reported numbers are not measured results of this implementation.
## Architecture
 [code] 
The observation estimator never receives latent predictions or simulator state. A planning-window stop prediction is separate from environment success. All model calls are accounted for, including unsuccessful episodes; retained latents are reused, but old Transformer KV caches are not carried across changed observations.
## Installation
The full model targets a **Linux NVIDIA GPU environment**. The reference training loop is single GPU with gradient accumulation. Dependency versions match the older OpenVLA stack; the pinned PyTorch build is not a validated Blackwell/RTX 50-series environment. No environment setup or package installation is performed merely by importing the package.
Create a dedicated environment using a verified interpreter. Commands below are instructions for a future run:
 [code] 
Use `.venv/bin/python` explicitly; shell activation is unnecessary. For core-only work, install `.[dev]`. Do not install project packages in a shared application runtime. On Windows, use a verified Python executable to create an environment and invoke `.venv\Scripts\python.exe`; full simulation/model execution is documented for Linux.
LIBERO is a separate optional dependency:
 [code] 
RoboTwin uses the `stable_2.0` interface pinned to `13c3c47ff4312dd62484bcd51be034af55c062d1`. Install its simulator/assets following its upstream instructions in a separate compatible environment, then install this package there. Current RoboTwin `main` uses a different policy integration stack. See benchmark protocols.
## Data and model artifacts
Bring your own permitted OpenVLA artifacts and benchmark data. Model loading defaults to `local_files_only: true`, with OpenVLA revision `47a0ec7fc4ec123775a391911046cf33cf9ed83f`. Set `model.backbone` to a complete local OpenVLA directory or make that revision available in the Hugging Face cache. The adapter loads pinned upstream model code with `trust_remote_code=True`; no weights are included here.
An end-to-end data pipeline is available:
 [code] 
`replay-labels` is optional when recorded predicate and observability traces are already available.