📈 GH Trending Explorer 🎲 Surprise me

← all repos

Tongyi-MAI / OpenGCUA

git clone https://github.com/Tongyi-MAI/OpenGCUA.git v0.1.2 · 1d ago

An open-source stack for generalist computer-use agents, combining code execution, GUI interaction, and cross-device control.

—
stars, 24h
+334
stars, 7 days · #173
+334
stars, 30 days · #799
334
★ total · +334% this week
24
forks · 1 open issues

Star history starts 2026-10-20; the chart appears after the second daily snapshot.

LanguageHTML
LicenseApache-2.0
Created2026-10-15 (5d ago)
Last pushtoday
Watchers2
OwnerOrganization

Links

README excerpt
### GPT-6 made computer use the frontier. We make it open.
News · Results · Demos · **Quick start** · SDK · Open stack · How it works · Contributing · Contact
**Computer use has reached a turning point.** GPT-6 Astra's demos offered an exciting vision of what
computer-use agents could make possible, and the focus is expanding from GUI-only interaction to both visual
and programmatic interfaces. The question has changed, from how well a model operates a GUI to how well a
system completes work on a real computer by whatever means works.
**We call such a system a generalist computer-use agent (GCUA).** It uses a computer in more ways than you do.
It reads the accessibility tree and calls app APIs, runs shell commands, and operates the GUI when it needs to.
With **code execution**, it writes GUI operations as code and combines them with all of these in one program,
with loops and conditions. It can also share the computer with you, working in background windows without
interrupting what you are doing.
**Yet frontier systems remain largely closed.** We therefore introduce **Open Generalist CUA (OpenGCUA)**, an
ongoing project to develop and open the full stack for GCUAs with the community, including model, system,
trajectories, efficiency optimization, and evaluation.
**What we open in this version:**
- 🧠 **OSWorld integration example** with reference screenshot-action
 and code-action agents for integrating the OpenGCUA design.
- 📜 **Evaluation trajectories** of the 8 frontier models, including GPT-6 Astra, Claude Opus 5,
 and Gemini 3.8 Flash.
- 💻 **macOS real-device harness** (preliminary), a menu-bar app that runs the agent on your own Mac.
- 🔌 **Cross-Device SDK** (preliminary), one programming interface for computers, browsers, and phones.
🗺️ See the detailed plan for our open stack.
## 🔥 News
- **[2026-10]** 🚀 The OpenGCUA research preview is out, with the tech report, the
 OSWorld integration example, the macOS real-device harness, the
 Cross-Device SDK, and evaluation trajectories of 8 frontier models.
## 📊 Results
We test the design on two fronts: OSWorld-v2 with eight frontier models, and real Macs with our real-device
harness.
- **Every model improves, including GPT-6 Astra, Claude Opus 5, and Gemini 3.8 Flash.** Over a GUI-only
 baseline with batched actions, the mean partial score rises from 45.3 to 64.9 (+19.6 points), and steps per
 task fall by 53% on average. GPT-6 Astra goes from 102 to 34 steps per task.
- **Weaker models gain the most.** GLM-5.3-Flash gains 35.2 points and DeepSeek-V4.1-Flash 29.8. The gap
 between the strongest and the weakest model shrinks from 39.1 to 18.0 points.
- **Code execution adds value beyond shell access.** With the same GUI and shell access, writing operations as
 code adds 6.1 points and cuts steps by 14.8% over a GUI+CLI agent (mean of three models).
With our real-device harness held fixed, on four macOS tasks Qwen3.8-Max used 19.4 times the reasoning tokens
of GPT-6 Astra and still completed fewer