obraia / gausify-orbit

Gausify Orbit 
 Turn orbit videos , including AI-generated ones, into Gaussian Splats . 
 Runs 100% locally, entirely in the browser, on WebGPU.
 Try it in the browser ·
 Run it locally ·
 How the web version works ·
 MIT License 
---
Drop in a video that circles around a subject (a person, a product, a sculpture,
or a clip from a video model), and Gausify Orbit extracts the sharpest frames,
optionally cuts out the subject, recovers the camera poses, trains a 3D Gaussian
Splat with live previews and exports a compact `.sog`/`.ply` you can share
with a link. It doesn't need CUDA or an install: everything runs in the browser
on WebGPU.
 [code] 
## Features
- **Orbit-aware frame selection:** samples N×K candidates across the chosen
 range and keeps the sharpest frame of each window, evenly spread around the
 orbit. Image sequences and folders work too.
- **Subject masking:** RMBG-1.4, SAM 2.1 video tracking with box/point prompts,
 a chroma key with despill for green and blue screens, or a mask video/image
 sequence you attach or generate with a ComfyUI workflow (an LTX 2.5 +
 IC-LoRA Alpha Gen one is in `assets/workflows/`). Masked training uses a random
 background, so no ghost silhouettes form around the subject, and edge colours
 are unmixed from the background, so no light fringe forms around hair. The
 optional **Reduce haze** keeps semi-transparent haze off the subject
 when it's seen from new angles.
- **Orbit-aware poses:** an incremental SfM built for orbits (AKAZE features,
 circular matching with long jumps on the GPU, bundle adjustment) that closes
 the orbit, plus inferred poses for frames it can't register. A second
 estimator, **COLMAP**, runs COLMAP's own SIFT and global mapper compiled to
 WebAssembly, with exact SIFT matching on the GPU
 (colmap-wasm, on npm): steadier
 on AI-generated orbits, at the cost of more memory.
- **GPU training without CUDA:** Brush,
 pinned to a fixed commit and patched, compiled to WASM + WebGPU. A live 3D viewer
 (Spark) reloads snapshots as training goes.
- **Optimized export:** PlayCanvas SOG v2 (lossless WebP textures, k-means SH
 palette), decimation that merges similar Gaussians, an opacity floor and
 SH band trimming. Typical output is 5–15× smaller than the raw `.ply`.
- **Mesh export:** a geometric fine-tune of the splat, depth maps fused
 into a sparse TSDF on the GPU, unseen holes closed, simplification and
 feature-preserving smoothing, then a UV atlas and the photos baked into its
 texture: a `.glb` that opens in the 3D view, Blender or three.js.
- **Share by link:** uploads the export to Cloudflare R2 and returns a private,
 expiring `/s/ ` link with a QR code. It opens in a PlayCanvas viewer with
 touch controls and **VR/AR** (WebXR).
- **Retouch with ComfyUI:** renders the splat along new camera paths,
 restores the clip with a video model in your ComfyUI and fine-tunes the splat
 on the restored frames.
- **Project packages:** save and reopen a whole project as a `.gorbit` file,
 even