AIMS-RAIL / RAIL

# RAIL: Rethinking Auditory Intelligence in Large Audio-Language Models with a CHC-Grounded Benchmark
**NeurIPS 2026**
Hongyu Jin 1,\* , Siyi Wang 1,\* , Yang Xiao 1,\* , Jiaheng Dong 1,\* , Shihong Tan 4 , Kaiyuan Peng 1 , Georgiana Juravle 2 , Shanquan Chen 3 , Gongping Huang 4 , Hong Jia 5 , Eun-Jung Holden 1 , James Bailey 6 , Ting Dang 1,† 
 1 The University of Melbourne, 2 Alexandru Ioan Cuza University of Iași, 3 The University of Hong Kong, 4 Wuhan University, 5 The University of Auckland, 6 Monash University
 \* Equal contribution. † Corresponding author.
Contact: Hongyu Jin (hongyuj1@student.unimelb.edu.au), Ting Dang (ting.dang@unimelb.edu.au)
Project page · arXiv · Paper on Hugging Face · Dataset on Hugging Face
**TL;DR:** RAIL evaluates large audio-language models the way cognitive science assesses human listeners. Grounded in Cattell–Horn–Carroll (CHC) theory, it organizes 5,306 audio questions into five core auditory capabilities and 32 subcapabilities, and compares 26 LALMs with a human baseline. Models do well on knowledge inherited from text pretraining, while fine-grained auditory perception and processing efficiency remain below human level.
| Capability | CHC | Subcapabilities | Samples | Audio (h) |
|---|---|---|---|---|
| Auditory Processing | Ga | 7 | 1,170 | 4.93 |
| Fluid Reasoning | Gf | 3 | 322 | 3.49 |
| Memory | Gsm + Glr | 6 | 1,000 | 13.0 |
| Processing Efficiency | Gs + Gt | 9 | 1,800 | 2.01 |
| Acquired Knowledge | Gc + Gkn | 7 | 1,014 | 7.2 |
| **Total** | | **32** | **5,306** | **30.6** |
## Repository layout
 [code] 
Clone with Git LFS to get the audio:
 [code] 
## Benchmark structure
Items are organised in three levels that follow the paper: 5 core capabilities (CHC broad
abilities), 32 tasks (CHC narrow abilities, named and coded as in the paper) and, where a task
has more than one design, subtasks. Each task is one file, `data/ / .jsonl`
(`U1/U9` is stored as `U1_U9`).
| Capability | Task (code) | Items | Answer format | Subtasks (items) |
|---|---|---:|---|---|
| Auditory Processing (Ga) | Speech Sound Discrimination (`US`) | 153 | multiple choice | Emotional Prosody Comparison (55); Confusable Word Identification (58); Minimal-Pair Discrimination (40) |
| | Resistance to Auditory Stimulus Distortion (`UR`) | 181 | multiple choice | Babble Noise (65); Band-Limiting (40); Background Music (36); Reverberation (40) |
| | Maintaining and Judging Rhythm (`U8`) | 160 | multiple choice | Beat Regularity Detection (40); Meter Identification (38); Tempo Change Detection (30); Tempo Comparison (52) |
| | Absolute Pitch (`UP`) | 200 | multiple choice | MIDI Pitch Identification (80); Note Name Identification (50); Relative Pitch Comparison (40); Frequency Estimation (30) |
| | Musical Discrimination and Judgment (`U1/U9`) | 102 | multiple choice | Aesthetic and Emotional Judgment (8); Genre Recognition (35); Instrument Identification (50); Musical Structure Analysis (9) |
| | Sound Localization (`UL`) | 182 | multipl