MCG-NJU / OneStreamer

OneStreamer 
 Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction 
 Xiangyu Zeng 1,* , Yuandong Yang 1,* , Zhiqiu Zhang 2,4,* , Yuhan Zhu 1,* , Xinhao Li 1,* , Qingyi Si 3,* 
 Dingyu Yao 6 , Changlian Ma 1,2 , Haoran Chen 1,2 , Xinyu Chen 1,2 , Yansong Shi 2,5 , Junhao Zhou 6 
 Yifei Li 9 , Jun Zhang 1,3 , Chuanyu Qin 6 , Chenxu Yang 6 , Xinlei Yu 7 , Kun Ouyang 8 
 Yuchen Shao 6 , Qianshan Wei 6 , Changhai Zhou 10 , Jun Gao 11 , Jiaqi Wang 3 , Limin Wang 1,† 
 1 NJU · 2 PJLAB · 3 JD · 4 SJTU · 5 USTC · 6 CAS 
 7 CUHK · 8 PKU · 9 THU · 10 FDU · 11 ZJU 
 * Equal contribution. † Corresponding author. 
 Overview ·
 Method ·
 Models and data ·
 Results ·
 Quick start ·
 Citation 
 Perceive the present. Remember the past. Respond at the right time. 
## :fire: Updates
- **2026/10/15**: 🔥🔥 The **source videos for OneStreamer-Core** are now available on Hugging Face!
- **2026/10/09**: 🎉🎉🎉 **OneStreamer** achieves **36.7** on InteractionBench with **native streaming**, ranking first among the reported open-source methods across both the **Native Streaming** and **Native Real-time Interaction System** categories!
## Overview
**OneStreamer** is a streaming video LLM that jointly learns to record evidence and respond through a shared proactive generation process. It preserves time-grounded factual memory before future questions are known, interprets incoming visual evidence, and decides when enough evidence is available to respond.
Built on Qwen3-VL-4B-Instruct, **OneStreamer-4B** achieves the best aggregate results among the compared methods on all eight benchmarks in the paper, spanning online perception, memory, and proactive response.
 Blue bars show OneStreamer; the lavender portions mark the Qwen3-VL baseline. Each panel uses its benchmark's metric. 
- **Reusable memory:** Proactive Hierarchical Caption Memory (PHCM) retains local details and completed-event summaries alongside a recent visual window.
- **Timely responses:** Proactive State Transition Learning (PSTL) preserves output anchors and selects representative state changes and persistence, outperforming dense state supervision with only **27.5%** of annotated state tokens supervised.
- **Evidence-aligned training:** OneStreamer-1M combines streaming captions, streaming QA, and cleaned open-source data into **over one million** interaction records.
## Method
**Proactive Hierarchical Caption Memory.** As video arrives, the model writes local-detail captions with ` ` and summaries of completed events with ` `. These time-aligned records remain in text history after their source frames leave the recent visual window, supplying context for later questions without revisiting historical visual features. During training, causal caption targets also supervise the interpretation of observed video prefixes.
**Proactive State Transition Learning.** PSTL keeps supervision at every output anchor and selects representative tokens for state changes and state persistence. Other s