Coral79 / PAMI-Code

# PAMI: Part Anchored Motion for Text to Human-Object Interaction Generation
 [🌐 Project Page] 
 [📄 Paper] 
This is the official repository for **PAMI**, a Part-Anchored Motion framework for text-conditioned full-body human–object interaction generation. Code, pre-trained checkpoints and evaluation scripts will be released here.
---
## 🚀 News
- **[Sep 2026]** Our paper is now on arXiv! Code and models are being prepared for release — stay tuned!
## 📦 Release Progress
- [ ] **Data Preparation**: scripts to convert InterAct into the part-anchored interaction representation.
- [ ] **Model Weights**: pre-trained checkpoints for PamiVAE, PamiGen and PamiRefiner.
- [ ] **Inference Code**: text-to-interaction generation with PamiGen + PamiRefiner.
- [ ] **Training Code**: training scripts for PamiVAE, PamiGen and PamiRefiner.
---
## 📄 Abstract
Text-conditioned full-body human–object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input text while remaining precisely coordinated over time. Most methods represent the human and object as separate trajectories and predict the global human–object couplings. Learning this complex, dynamically changing relationship implicitly, however, often yields object drift, missed contact, and penetration.
We introduce **PAMI**, a Part-Anchored Motion framework for Interaction generation. Inspired by the classic Hough Transform, our key idea is to localize object motion by letting body-part anchors vote for it: we express object motion relative to multiple body-part anchors and use **PamiVAE** to learn an interaction latent space, decoding frame-wise weights that aggregate these part-specific votes. Building on this representation, PAMI generates interactions in a coarse-to-fine hierarchy. **PamiGen** first generates a coarse human–object interaction from text in this structured latent space, and **PamiRefiner** then recursively resolves fine-grained contact geometry using a hybrid surface-sensing representation, combining long-range probes that capture overall body-part influence with short-range sensors that resolve detailed contacts near the object surface.
Experiments on InterAct show that PAMI generates more faithful interactions and more accurate human-relative object motion than previous methods, achieving 14.5% higher contact recall than the previous state of the art. Extensive ablations validate the contributions of both the part-anchored voting representation and hybrid surface-sensing refinement.
---
## ⚙️ Installation
*Coming soon.* The environment file and setup instructions will be added together with the code release.
---
## 🗂️ Data Preparation
 Click to expand 
PAMI is trained and evaluated on **InterAct**, a unified collection of text-annotated human–object interaction mocap datasets (OMOMO, CHAIRS, NeuralDome, IMHD, BEHAVE, GRAB, InterCap) with SMPL-H / SMPL-X bodies and object meshes.
1. Download InterAct following the instructions of the official r