SinapisAI / collective-model-intelligence

# Collective Model Intelligence
**Toward Collective Model Intelligence: Characterizing and Coordinating Models**
**Tech Report (arXiv)** · Results · Try Mio · Open-Source Dataset · 中文
## Mio 1.0 Lite
**Mio (Models in One)** organizes complementary models to work together, choosing which model to call as task needs and remaining budgets change. **Mio 1.0 Lite** applies this approach to cost-efficient models from China.
Across nine equally weighted public benchmarks, Mio 1.0 Lite achieves an average score of **77.15/100** at **0.0621 CNY/request**, the highest average score and lowest average cost among 13 compared methods. Compared with the strongest baseline by average score, Mio scores **6.35 points higher** at **94.0% lower average cost**. Full benchmark results →
## From individual models to collective intelligence
**Collective Model Intelligence** aims to improve task outcomes and resource efficiency by understanding, reusing, and coordinating capabilities across an evolving population of independently developed models. The key is complementarity: a specialist can supply a missing capability, and different models can contribute at different stages as task needs and remaining cost and latency budgets change.
## System Architecture Overview
- **Model Eval** builds capability profiles through query-level score estimation and cross-domain prediction.
- **Mio (Models in One)** organizes complementary models and selects or coordinates them using task state and remaining budgets.
- **Mos (Model Serving OS)** schedules and executes evaluation and application calls, and manages deployment and model-file distribution for self-managed models.
**Model Space** provides shared model identities and resource references, linking capability evidence, candidate pools, and execution.
## RobustRouteBench
RobustRouteBench contains **35,072 instances** across General Text (63 candidate models), Multimodal (45), and Chinese (10): **14,590 training**, **1,500 validation**, and **18,982 test requests**. The test split directly matches the report's evaluation population.
Use **Python 3.10+**. The examples need no model API key.
 [code] 
Other scenarios: `--scenario multimodal` or `--scenario chinese`.
Replay the **65 prediction runs across 13 methods** on the official test split using the report's nominal-budget nAUC:
 [code] 
Dataset and protocol · Reproduction guide · Comparison methods
## Citation and license
 [code] 
Citation metadata: CITATION.cff. Code uses Apache-2.0, with MIT-licensed ORBIT components. The dataset follows its upstream terms; report materials and embedded logos retain their attribution.