OpenDCAI / DataFlex

# DataFlex
 Data Select · Mix · Reweight — Right in the LLM Training Loop
🎉 If you like our project, please give us a star ⭐ on GitHub for the latest update.
简体中文 | English
## 📰 1. News
- **[2026-09-25] 🎉 DataFlex has been accepted to the NeurIPS 2026 Evaluation & Dataset Track!** Our paper, *DataFlex: A Unified Benchmark and Evaluation Platform for Data-Centric Training of Large Language Models*, will appear at **NeurIPS 2026**.
- [2026-04-04] 🎉 Our technical report ranked #1 on the Hugging Face Daily Papers leaderboard for that day.
- [2026-03-17] We now support gradient computation under DeepSpeed ZeRO-3, enabling training and analysis of larger-scale models.
- [2025-12-23] 🎉 We’re excited to announce the first Data-Centric Training System DataFlex, is now released! Stay tuned for future updates.
## 🔍 2. Overview
**DataFlex** is an advanced dynamic training framework built on top of LLaMA-Factory. 
It intelligently schedules training data during optimization and integrates several difficult-to-reproduce repositories into a unified framework. The system provides reproducible implementations of **Data Selection**, **Data Mixture**, and **Data Reweighting**, thereby improving both experimental reproducibility and final model performance.
DataFlex integrates seamlessly with LLaMA-Factory, offering researchers and developers more flexible and powerful training control. For goals and design philosophy, please refer to DataFlex-Doc.
We summarize repositories related to Data Selection, Data Mixture, and Data Reweighting.
❌ indicates that no official repository is available;
✅ indicates that an official repository is available;
⚠️ indicates that an official repository exists but contains issues.
- **Data Selection**: Dynamically selects training samples according to a given strategy (e.g., focus on “hard” samples). The data selection algorithms are summarized as follows:
| Method | Category | Requires Model-in-the-Loop? | Official Repo |
|:------:|:--------:|:---------------------------:|:-------------:|
| **LESS** | Gradient-Based | ✅ Yes | ⚠️official code |
| **NICE** | Gradient-Based | ✅ Yes | ⚠️official code |
| **Loss** | Loss-Based | ✅ Yes | ❌ |
| **Delta Loss** | Loss-Based | ✅ Yes | ❌ |
| **NEAR** | Data Distribution-Based | ❌ No | ❌ |
| **TSDS** | Data Distribution-Based | ❌ No | ✅official code |
| **Static** | No Selection | ❌ No | ❌ |
| **Random** | Random Sampling | ❌ No | ❌ |
- **Data Mixture**: Dynamically adjusts the ratio of data from different domains during training. The data mixture algorithms are summarized as follows:
| Method | Category | Requires Model-in-the-Loop? | Official Repo |
|:------:|:--------:|:---------------------------:|:-------------:|
| **DOREMI** | Offline Mixture | ✅ Yes | ⚠️official code |
| **ODM** | Online Mixture | ✅ Yes | ⚠️official code |
- **Data Reweighting**: Dynamically adjusts sample weights during backpropagation to emphasize data preferred by the model. The data reweighting algorithms are summarized