TencentARC / GameHorizon

Yiran Wang 1,*,† , Xingyilang Yin 1,2,6,* , Junfu Pu 1,* , Guangzhi Wang 1,* , Kaifeng Li 2 , 
 Mingyu Ouyang 1,3 , Huiqiang Sun 1,4 , 
 Lingen Li 1,5 , Cheng Cheng 1 , Wangbo Yu 1 , 
 Honghao Chen 1 , Xiaodong Cun 1,2,✉ , Chi-Man Pun 6 , Zhiguo Cao 4 , Ying Shan 1 
 1 ARC Lab, Tencent   2 GVC Lab, Great Bay University   3 NUS   4 HUST   5 MMLab, CUHK   6 University of Macau 
 † Project Lead    * Equal Contribution    ✉ Corresponding Author
---
## 📢 News
- **[2026.09.22]** 📄 The paper and project page of GameHorizon Suite are released!
- We will progressively release the **code**, **benchmark**, and **data** starting around **2026.10.25**. Stay tuned! ⭐
## 🚀 Open-Source Plan
We are actively preparing the release of our code, benchmark, and data. Star the repo to follow our progress.
- [x] 🌐 Project page
- [x] 📄 Technical report / paper 
- [x] 🏆 Public leaderboard
- [ ] 🏗️ GameHorizon-Annotator — annotation toolkit & pipeline
- [ ] 📊 GameHorizon-Bench (offline) — MCQ suite & evaluation code
- [ ] 📊 GameHorizon-Bench (online) — online task environments
- [ ] 🎯 GameHorizon-Data — 5,000-hour AAA gameplay corpus
## 📖 Overview
We introduce GameHorizon , a large-scale data and evaluation suite spanning multiple horizons and AAA games. It serves as a unified yardstick across a broad range of model types. It consists of three key components. GameHorizon-Annotator automatically produces a three-level pyramid of short-horizon operations, medium-horizon goals, and long-horizon strategies. GameHorizon-Data contains 5,000 hours of gameplay across 21 game titles, with temporally aligned videos, actions, and multi-horizon instructions. GameHorizon-Bench provides reproducible offline and stepwise online testing. The offline track contains thousands of standardized MCQs across three primary tasks and diagnostic variants. The online track tests order-dependent causal and order-flexible thematic tasks via the verifiable subtasks for failure localization.
 Component 
 What it is 
 Highlights 
 🏗️ Annotator 
 A scalable and automated annotation pipeline 
 L1 → L2 → L3 instruction pyramid 
 🎯 Data 
 A large-scale AAA gameplay corpus 
 5,000 hours · AAA-focused · 21 titles · temporally-aligned videos, actions & multi-horizon instructions 
 📊 Bench 
 Reproducible offline + stepwise online track 
 5,000 offline MCQs (3 primary + 10 variant tasks) · 20 online tasks / 62 subtasks · 47 models benchmarked 
## 🏆 Leaderboard
### Offline Results — Primary Tasks (T1 / T2 / T3)
Accuracies are reported as percentages. **Overall** is the mean across the three tasks. 
**Bold** = best, underline = second-best. Models are grouped into four tiers by Overall score.
 Average (all 44 models) — T1 57.3 · T2 65.1 · T3 71.6 · Overall 64.7 
 🥇 Tier 1 · Rank 1–11 
 # Model T1 T2 T3 Overall 
 🥇 GPT-6-Astra 69.4 79.6 91.5 80.2 
 🥈 Gemini 3.8 Flash 65.9 81.2 84.8 77.3 
 🥉 Gemini 3.7 Flash 66.2 80.1 83.9 76.7 
 4 Gemini 3.6 Flash 62.3 79.1 84.6 75.3 
 5 GPT-5.6 Sol 65.3 77.5 81.5 74.8 
 6 Kimi-K3 6