reality-opened / openreality

The Agentic OS for any hardware 
 Open Reality gives your AI agent the ability to use any hardware. Vision + Action.
 Website ·
 Get started ·
 Things you can ask ·
 Self-host ·
 MCP docs ·
 Contributing 
---
 Demo v1: a real Claude Code (Opus) session against the built-in offline simulator, beside a synchronized scene panel. Video: docs/demo/demo-video-v1.mp4 . Rendered from docs/demo/ with VHS . 
 Live reconstruction from a handheld video: the camera path and the 3D scene build up together. Scan demo and the initial technical idea: VGGT-SLAM by Dominic Maggio, Hyungtae Lim and Luca Carlone (MIT SPARK Lab). Open Reality's reconstruction core builds on their work. 
## Utilities
| | |
|---|---|
| 🎥  **Video in, 3D scene out** | Upload a phone video. A few minutes later you have a persistent 3D scene. |
| 📏  **Measurement** | Distances and angles between any points. Numbers are only called metres after you calibrate with real distance; otherwise relative.|
| 🧭  **Path planning** | Plan a route through the scanned free space to an object or a point. |
| 🤖  **Robot-training exports** | Turns a scan into LeRobot / GR00T style datasets or an Isaac Sim or MuJoCo scene (hosted service). |
| 🕵️  **Scene agents** | Server-side agents that survey, label, and answer questions about a scene. |
| 🛠️  **42 tools for your AI** | Everything is exposed through MCP. |
| 🧪  **Offline simulator** | A mock backend fakes the entire workflow with fixture data, so you can develop and demo with no account and no GPU. |
| 🏠  **Self-hostable** | The full server runs on your own GPU box or your own Modal account, no account with us needed. |
## Quickstart
Add the tools to your AI assistant:
 [code] 
 Claude desktop / Cursor (click to expand) 
Add this to `claude_desktop_config.json` (Settings → Developer → Edit Config)
or `~/.cursor/mcp.json`:
 [code] 
 Claude Code plugin / Claude desktop one-click extension (click to expand) 
Claude Code plugin (the MCP server plus the workflow skill in one step):
 [code] 
Claude desktop extension: download `openreality-mcp- .mcpb` from the
latest release
and double-click it, or drag it into the Claude desktop window. No terminal needed.
Then sign in once (opens your browser, stores a revocable API key on your machine):
 [code] 
Scan a room with your phone at open-reality.io, or just ask
your assistant to upload a video file. Full per-client setup:
open-reality.io/mcp.
## Examples
Once connected, talk to your assistant like this:
> "Upload `~/Videos/kitchen.mp4` and reconstruct it."
> "What objects are in my latest scan, and how big is the room?"
> "The counter edge to the window is 2.4 m. Calibrate the scene, then measure the couch."
> "Plan a path from the door to the desk and describe it."
> "Export this scan as robot-training data and save the zip locally."
## Self Hosting
The whole workflow is self-hostable. Read
`server/docs/self-hosting.md`; the short version:
 [code] 
Self-hosted servers need no account: a single token printed at fi