appletea233 / OneSearch-VL

Unified Multimodal Deep Research Agent for Image and Video 
 Unified Tool Set  ·  VGEG-Centered Data Engine  ·  Rubric Reward  ·  Multi-Image / Video Benchmarks 
## 📑 Table of Contents
- 📖 Introduction
- 🗺️ Overview
- 🍭 Method Overview
- 📊 OneSearch Benchmarks
- 🏆 Main Results
- 🔎 Case Study
- 🤗 Models & Datasets
- 📁 Repository Layout
- 🛠️ Getting Started
- 🏋️ Agentic SFT
- 🚀 Agentic RL
- 🔬 Inference & Evaluation
- 🙌 Acknowledgements
- 📄 Citation
- ⚖️ License
---
## 📖 Introduction
**OneSearch-VL** is a unified multimodal deep research agent for **single images,
multi-image collections, and videos**. It locates visual clues, retrieves external
knowledge, and combines evidence into supported answers.
At its core is the **Visually Grounded Evidence Graph (VGEG)**, which connects
visual anchors, source-supported facts, and answer-producing operations. This
shared structure supports three parts of our approach:
- **Data construction:** a VGEG-centered data engine generates and verifies
 research questions and expert tool-use trajectories.
- **Evidence supervision:** the **Evidence-aware Visual-Grounded Rubric reward
 (EVGR)** supervises evidence traceability and visual grounding during RL.
- **Evaluation:** **OneSearch-MI-Bench** and **OneSearch-Video-Bench** evaluate
 how agents retrieve and compose evidence across six research operations.
---
## 🗺️ Overview
Starting from **Qwen3-VL-8B**, OneSearch-VL learns a shared research policy through
joint SFT and RL across all three visual input types. The agent combines visual
inspection, image and text search, and video frame selection in a common tool
environment. VGEGs provide supervision and evaluation references; the agent
receives the visual input, question, and tools and gathers its own evidence.
This repository provides SFT, RL, and
inference and evaluation code. Model weights and data
are available under OneSearch-VL on Hugging Face.
---
## 🍭 Method Overview
### VGEG-Centered Multimodal Data Engine
Our data engine discovers visual anchors, connects them to verified web facts,
and constructs compositional questions with explicit evidence dependencies.
Expert trajectories are filtered for answer correctness and process quality.
Together with adapted single-image data from
OpenSearch-VL, this produces
**OneSearch-VL-SFT-110K** and **OneSearch-VL-RL-10K** for unified training.
*VGEG-centered data construction, from visual sources to verified research tasks and expert trajectories.*
### Evidence-aware Visual-Grounded Rubric Reward (EVGR)
EVGR uses VGEG-derived rubrics to evaluate whether a rollout obtains the required
facts (**evidence traceability**) and uses the correct images, regions, or video
frames (**visual grounding**). Combined with answer and query rewards, it
encourages the agent to ground its answers in the evidence it actually gathers.
*EVGR supervises evidence traceability and visual grounding during reinforcement learning.*
---
## 📊 OneSearch Benchmarks
**OneSearch-MI-Bench**
contains **30