llros007 / lerobot_android

**English** | 中文
# lerobot_android
The phone does not need root and can be used as it is. It uses the phone's built-in front camera. A USB camera and the servo control board plug straight into the phone over USB OTG to control a mobile robot.
The phone sits on the robot and shows the eyes. The follower serial adapter and the wrist camera plug into the phone. The PC records the front camera, the wrist camera, the arm, and the base together:
## What it does
- Teleoperate one arm from an SO-101 leader on the PC.
- Drive the differential-drive base from the keyboard while the arm follows the leader.
- Record one episode: joint positions, `x.vel`, `theta.vel`, and the phone cameras, in the normal LeRobot dataset format. Recording at 720p and 30 fps works now, using two USB cameras.
## How it works
The phone app owns the follower USB serial port. A 30 Hz loop on the phone reads the servos and writes the latest goal, the same pattern as a LeKiwi host. The PC does not wait for a Wi-Fi round trip inside that servo tick. It talks to the phone with `host:// ` (commands on port 9101, state on port 9102). Cameras are MJPEG over HTTP on port 8080. The leader arm stays on the PC.
The phone encodes each camera frame as JPEG and streams MJPEG, instead of sending raw pixels. The PC reads that stream in the background and the control loop uses only the newest frame, so a late frame does not stall the 30 Hz arm loop. The socket timeout is 15 seconds because the first open of a USB camera takes a few seconds.
Putting the serial link on the PC means every tick reads and writes across Wi-Fi. Each tick waits for a round trip, and the loop falls below 30 Hz. The read and write now stay on the phone, next to USB. The PC only exchanges the newest goal and the newest state, so Wi-Fi delay sits outside the servo period.
`lerobot/` is the Hugging Face LeRobot tree at tag **v0.6.1** (commit `7e241bd63`), plus the XLeRobot robot, the phone host client, and an MJPEG camera. Dependencies match the upstream `pyproject.toml`. Python **3.12** is required. This folder does not include the upstream Git history. `android/` is the phone app.
A debug APK is in `release/lerobot-android-debug.apk`.
## Why a phone
1. The phone already has a screen, cameras, an NPU, an IMU, GNSS, 5G, Wi-Fi, and a compass. Those parts do not have to be bought and wired one by one.
2. Those sensors are not published yet. See the list below.
## To do
- [ ] ROS 2: publish the cameras, joint state, and base velocity.
## 1. Computer
 [code] 
`lerobot[feetech]` is the same hardware extra upstream uses for Feetech STS servos. The rest of the environment matches the official package.
## 2. Phone
Build and install `android/` (JDK 17). On the phone, turn on OTG, allow camera, notification, and USB permission, then start the foreground bridge service. If the camera and the serial adapter share one OTG port, use a USB hub.
The follower arm's USB serial adapter plugs into the phone. The first seria