Video-to-3D Reconstruction
Upload a video and reconstruct a 3D scene from it using a multi-agent ML pipeline.
Status: planned. This feature is not currently in the codebase (prototype removed 2026-07).
The sections below describe the intended design. No video store, panel, or startVideoReconstruction Cloud Function currently ships.
Pipeline Stages
| Stage | ML Model | Purpose |
|---|---|---|
segment | SAM2 | Segment video frames into objects |
depth | Monocular depth estimation | Estimate per-pixel depth |
classify | Object classification | Identify what each segment is |
rig | Auto-rigging | Add armatures to characters/vehicles |
compose | Scene composer | Place objects in 3D space |
Job Lifecycle
uploaded → segmenting → depth → classifying → rigging → composing → done
└→ failed
Each stage would update a videoJobs/{jobId} Firestore document, and a VideoPanel would subscribe via onSnapshot to show real-time progress bars.
UI
A VideoPanel would provide:
- File input for video upload
- Job list with per-stage progress indicators
- "Reconstruct" button to kick off the pipeline
- Result GLB download when complete