POInfra-AI / POInfra

POInfra: Policy Optimization for Generative Models 
**POInfra focuses on policy optimization for generative models.** We study how diffusion and flow models can serve as policies, how these policies can improve through reinforcement learning, and how additional computation at inference time can improve their decisions.
Our research began with **D2PPO**, which combines dispersive regularization for diffusion policy pretraining with PPO fine-tuning. The research extends to efficient one-step policies, coordinated multi-agent generation, online fine-tuning, and inference-time guidance and planning. The common goal is to turn expressive generative models into effective decision-making policies.
**PO** stands for **Policy Optimization**, and **Infra** reflects our goal of building reusable infrastructure around this research. This repository brings together the methods and resources. Available implementations currently run through their individual project repositories.
## Models and datasets
Browse the POInfra collection on Hugging Face, or open a project resource directly below.
| Project | Models | Datasets |
| --- | --- | --- |
| **OGPO** | Hugging Face | Hugging Face |
| **CoFlow** | Hugging Face | Hugging Face |
| **MA-FPPO** | Hugging Face | — |
| **G2MAF** | Hugging Face | — |
| **MA-WAM** | Hugging Face | — |
## Video overview
A two-minute introduction to POInfra, with real robot demonstrations and multi-agent environments.
https://github.com/user-attachments/assets/c93f9101-ff76-4da4-84df-8372ea182742
## What's NEW!
- [2026/09] 🔥 **POInfra** is launched, bringing together our research on policy optimization for generative models, beginning with D2PPO.
- [2026/09] 🔥 **MA-FPPO** is now available on arXiv. The project page includes policy rollouts and a presentation of multi-agent flow pretraining and online fine-tuning. Project · Paper · Code · Models.
- [2026/07] 🔥 **G2MAF** explores test-time gradient guidance for multi-agent flow policies. Project · Paper · Code.
- [2026/07] 🔥 **MA-WAM** explores multi-agent world-action models for test-time planning. Project · Paper · Code.
- [2026/05] 🔥 **CoFlow** introduces coordinated few-step flow for offline multi-agent decision-making. Project · Paper · Code.
- [2026/09] 🔥 The updated **OGPO** preprint (v2) is available. The work is now accepted at **ACM MM 2026**. Project · Paper · Code.
- [2025/10] 🔥 **DM1** introduces MeanFlow with dispersive regularization for one-step robotic manipulation. Project · Paper · Code.
- [2025/08] 🔥 **D2PPO** introduces diffusion policy optimization with dispersive loss. The work is now accepted at **AAAI 2026**. Project · Paper · Code.
## Design Philosophy and Research Directions
POInfra aims to bring embodied intelligence into practical use, enabling robots to perform useful work in factories and homes and create tangible value. We focus on a central question: **How can generative models move from learning existing behavior to becoming decision-making policies that c