ja1chen / actell

# iPhone Use · Phone Agent · Actell
English | 简体中文
**An early technical prototype built through vibe coding.** Actell explores an iPhone agent that captures the screen and sends touch input through CoreDevice. VPN loopback is the default route; ESP32 USB NCM (Network Control Model) is optional. Long-running background operation and production readiness are not guaranteed.
https://github.com/user-attachments/assets/2aefc1a7-bdf1-42e7-83b9-51bd879256c8
## Capabilities
- Conversations with configurable providers and models, streaming replies, readable reasoning supplied by providers, attachments and imported text skills.
- Screenshot, tap, drag and wait tools. Tool calls run in order; uncertain inputs are never automatically replayed. Ordinary chat does not require a device connection.
- VPN and NCM share an App-owned system pairing identity and CoreDevice backend. Controlled device tests have demonstrated screenshots and actual taps; complete tasks, drag and long-running background operation need separate acceptance.
- Optional audio, picture-in-picture and location background sources. New installations enable audio and residency; location and standby Live Activities start off. Saved choices remain effective, and system execution limits still apply.
- An optional Muse companion interface, with per-device permissions and a separate execution runtime. A general MCP service remains future work.
## Setup
Use **Xcode 27 and iOS 27 or later**. Physical installation requires your own valid signing configuration. Configure a compatible provider and credentials for model tasks; screenshots and explicitly sent attachments are sent to that provider.
1. Enable Developer Mode and local network access. Complete the App's pairing flow in the system UI.
2. Prepare a matching Personalized Developer Disk Image, usually through Xcode. The backend checks availability; it does not download, personalize or mount images.
3. In Settings → Phone Control, select the connection route and open Connection & Validation:
 - VPN loopback, the default: enable an external VPN with local packet reinjection. Default interface/peer addresses are `10.7.1.1` / `10.7.0.1`; match your VPN configuration. The App does not manage the VPN. Use Wi-Fi when connecting; cellular operation and network switching have separate limitations.
 - ESP32 NCM: connect an ESP32-S3 with the current firmware through its native USB data port. Check the hardware conditions.
4. Add a provider, select a model and create a conversation. Control tools check connection and authorization when invoked; the route cannot change during execution.
VPN apps with built-in loopback can share the same connection with regular proxy traffic. Compatibility has been tested with LocalDevVPN, Shadowrocket and Clash; interface addresses and required settings vary. See the route guide.
## Build and test
Install Rust with the `aarch64-apple-ios` and `aarch64-apple-ios-sim` targets. The backend uses a locked dependency graph; set `CORE