Human-Agent-Society / reef

Infrastructure for continually self‑improving agents 
English | 中文
Reef is the first open-source infrastructure for continual self-improving agents.
It connects agent inference, feedback, learning, and versioned delivery. Use it
to train model weights with Slime and SGLang, or improve an agent's harness, including its prompts, rules, and skills.
**🚀 Get started |
🗺️ Roadmap |
🗓️ Office Hours |
📣 Launch post |
💬 Join Discord |
📱 Join WeChat Group**
## 🎯 When to use Reef
Use Reef when you want your agent to keep improving simply by learning from how you interact with your agent.
| Your goal | Learning path | What you need |
|---|---|---|
| Keep getting stronger model designed for you | Model weight training | A trainable model, a supported GPU stack, and feedback your recipe can use |
| Get your harness to self-improve | Harness optimization | A model endpoint, representative tasks, and an evaluator; no local training GPUs |
| Scientific discoveries | Test-time training | An execution environment, a correctness checker, and a measurable objective |
## 🧩 How Reef fits your stack
| Ability | Inference engine (vLLM, SGLang, …) | RL training framework (Slime, veRL, AReaL, …) | **Reef** |
|---|:---:|:---:|:---:|
| Serves live traffic | ✅ | ❌ | ✅ |
| Trains weights | ❌ | ✅ | ✅ |
| Version management | ❌ | ❌ | ✅ |
| Stays live through updates | ❌ | ❌ | ✅ |
| Evolves beyond weights (skills, harness) | ❌ | ❌ | ✅ |
## 🔄 How it works
Reef processes each learning cycle in four steps. The table also shows which
modules implement each step.
| Step | What happens | Where it lives |
|---|---|---|
| **1 · Serve** | Serve agent requests and record interactions. | `service/` — agent requests and interaction records `runtime/` — inference and artifact updates |
| **2 · Observe** | Match feedback to recorded interactions. | `storage/records.py` — stored interactions and feedback `record2record/` — feedback matching and eligibility |
| **3 · Grow** | Produce an update from eligible records. | `recipe/` — recipe integration `train/` — the training pipeline, batches, and update jobs |
| **4 · Commit** | Apply the configured selection policy and publish accepted updates. | `evaluation/` — candidate evaluation `artifact/` — version history `surface/` — artifact delivery |
## 📦 Installation
> 💡 **Note**
>
> Reef's artifact and checkpoint functionality requires the `git-lfs` system
> package. Reef initializes Git LFS locally for its artifact repositories.
We recommend uv for managing packages, and the
commands below use it.
### From PyPI
 [code] 
### From source
 [code] 
Use the source checkout for development and for the training examples below.
## 🔧 Using Reef
Reef supports two learning surfaces: model **weights** and agent **harnesses**.
The deployment's recipe determines which surface its scenarios update.
As a minimal example, start Reef as a pure inference server:
 [code] 
### Weight-training deployment
#### Start the deployment
The following example starts the SAO (arX