camenduru / TostAI-Sprite-Sheet-Studio

🐣 Please follow me for new updates: https://x.com/camenduru 
🔥 Please join our discord server: https://discord.gg/k5BwmmvJJU 
🥳 Please become my sponsor: https://github.com/sponsors/camenduru 
🍞 TostUI repo: https://github.com/camenduru/TostUI
#### 🍞 Tost AI - Sprite Sheet Studio
https://github.com/user-attachments/assets/7aec9682-719a-45fd-842c-abe9877205ea
Two tools in one server:
1. **Generator** — turn a raw video into a game-ready sprite sheet:
 **VRMBG-3.0 video matting (or an optional mask video) → optional backdrop
 repair → tight auto-crop → packed grid sheet + sidecar JSON + preview player
 + GIF**, with an invariant gate that checks the result instead of trusting it.
2. **Editor** — open any existing sheet (from a run, from `walk/`, from
 `sprites/`, or any PNG + sidecar) and fix it: align the pivot, repair the
 matte, key out a colour, repaint, re-grid, re-time, re-export.
## Run it
 [code] 
- **http://127.0.0.1:8765** — the generator
- **http://127.0.0.1:8765/editor** — the editor
Options: `--host`, `--port`.
Requires the Python that has the ML stack (`torch`, `cv2`, `transformers`, `PIL`,
`numpy`, `fastapi`, `uvicorn`) — on this machine that is the 3.13.9 install above,
not the sandboxed runtime. `ffmpeg` is optional and only used by the helper
scripts.
### Docker
The image is self-contained: it carries the model and the three pixel ops.
1. **Install Docker**\
 Download Docker Desktop (Windows AMD64)
 and run it.
2. **Update the container (optional)**
 [code] 
3. **Run the container**
 [code] 
 *Requires NVIDIA GPU (Min 4GB VRAM)*
4. **Open app**\
 Go to: http://localhost:8765 (generator) or http://localhost:8765/editor (editor)
**Build.** The one token arrives as a **secret mount sourced from env var** —
never `--build-arg`, because a build-arg token is visible in
`docker history --no-trunc`. It lives in `.env`:
 [code] 
- `HF_TOKEN` — the VRMBG-3.0 repo is gated; the model download 401s without it.
- `CACHEBUST` — the app clone is deliberately unpinned (always latest source), and
 a cached `git clone` silently re-serves the first snapshot. Bumping this forces
 a re-clone. **Omit it and you get stale code with no warning.** The commit that
 landed is written to `.sprite_rev` inside the image.
**Run:**
 [code] 
- `--gpus all` is **not optional** — without it `torch.cuda.is_available()` is
 `False` and matting falls back to CPU.
- **http://127.0.0.1:8765** — the generator
- **http://127.0.0.1:8765/editor** — the editor
**Check it actually came up.** The healthcheck probes the app:
A from-scratch rebuild (`--no-cache`) re-downloads ~7.4 GB, including the 845 MB
model.
## The workflow
1. **Input** — type a path, use *Browse…*, or drag a file onto the drop zone
 (uploads land in `uploads/`). *Read video* shows frame count, fps, size, a
 **playable preview of the clip**, and the valid column counts.
2. **Background removal** — VRMBG-3.0, autoregressive along time. Pick the
 inference size, a frame range and a step.
 - **…or a mask video