RUC-NLPIR / Agent-Reflex

Agent-Reflex: Teaching Language Agents to Act Reflectively 
##### If you like our project, please give us a star ⭐ on GitHub for the latest update.
*A training framework that elicits **proactive reflection** in tool-using LLM agents: noticing noisy tool feedback and user corrections, and recovering from them.*
Demo · Overview · Framework · Results · Installation · Data · Quick Start · Citation
## 📣 News
* **[2026-10]** 🤗 The **ReSFT data (54,663 trajectories)** and **ORPO tasks (2,550 with skills)** are now available on Hugging Face: SFT-54K · RL-2K.
* **[2026-10]** 🤗 The **Agent-Reflex-8B** checkpoint (Qwen3-8B, ReSFT + ORPO) is released.
* **[2026-10]** 🚀 Full codebase released: the RSD skill library (1,170 meta-skills), ReSFT and ORPO training recipes, and evaluation on six benchmarks.
* **[2026-10]** 📄 Our paper **Agent-Reflex: Teaching Language Agents to Act Reflectively** is now available on arXiv and Hugging Face Papers.
## 🎬 Demo
### Agent-Reflex in One Minute
https://github.com/user-attachments/assets/281af1d3-843e-4496-b232-58fee951cefd
 🎥 Download the 1080p video (60 seconds) · 💻 original task interactive replay (open locally in a browser) 
## 💡 Overview
**Agents are trained in idealized environments but deployed into noisy ones.** Two gaps follow:
* **(1) No noisy feedback.** Real MCP tools time out, reject permissions or return malformed results, yet training environments return clean feedback, so agents never learn to detect and recover from failures.
* **(2) No user correction.** Real users interrupt and correct the agent mid-task, yet user simulators rarely issue such corrections.
 Figure 1 (a, b) · The two gaps. Idealized interaction (a) vs. real-world interaction with user corrections and noisy feedback (b). 
 Figure 1 (c, d) · The cost of noise. Strong agents lose success rate under user or tool noise (c); tool errors fall into eight recurring families (d). 
**Agent-Reflex** treats reflection as a learnable meta-skill: it mines skills from failures in noisy environments, distills them into supervised data and reinforces them with noisy agentic RL. On Qwen3-8B it improves six agent benchmarks and stays robust under noise.
## ✨ Agent-Reflex Framework
### Stage 1 · Environment Reflection Skill Discovery (RSD)
Build failure-aware executable environments by perturbing tool returns with real-world failure families, and mine verifier-matched success / failure pairs into an offline library of **1,170 reflection meta-skills** over **557 environments**. Environment synthesis and the environment infrastructure follow Agent-World.
### Stage 2 · Reflective Supervised Fine-Tuning (ReSFT)
Convert each skill into verifier-validated **self-reflection (SR)**, **skill recovery (KR)** and **user-correction (UR)** data, with skill prompts stripped, and internalize them by supervised fine-tuning.
### Stage 3 · On-Policy Reflective Policy Optimization (ORPO)
Combine Noisy Agent RL with same-token, skill-guided on-policy distillation (OPD) to provide