Extensions/ComfyUI-SplatCam
ComfyUI Extension

ComfyUI-SplatCam

Turn one image into a camera you can fly — a live 3D Gaussian-splat viewport inside a ComfyUI node, with mask, depth, and camera-move recording for video-model camera reference.

By tahabahrami·Created 21 days ago·Updated 21 days ago· 1
tahabahrami/ComfyUI-SplatCam
Nodes1
On cloudLocal install
Category3d/SplatCam
Stars1
Updated21 days ago

Nodes (1)

SplatCam · Image → 3D Camera
3d/SplatCam
Readme
<p align="center"> <img src=".github/assets/banner.png" alt="SplatCam — One image in. Camera control out." width="100%" /> </p> <h1 align="center">SplatCam</h1> <p align="center"><b>Turn one image into a camera you can fly.</b></p> <p align="center"> A single ComfyUI node that builds a 3D Gaussian splat from your image, opens a live viewport <i>on the node itself</i>, and records any camera move you fly — as an RGB, mask, or depth video — ready to drive video models as a camera-motion reference. </p> <p align="center"> <a href="#install">Install</a> · <a href="#quick-start">Quick start</a> · <a href="#tutorial">Tutorial</a> · <a href="#the-camera-reference-recipe">Camera-reference recipe</a> · <a href="#privacy--cost">Privacy &amp; cost</a> · <a href="#faq">FAQ</a> </p>

What it does

<p align="center"> <img src=".github/assets/tutorial.gif" alt="SplatCam tutorial: install, wire one image, build the splat, fly the camera, record the depth pass, save the video" width="100%" /> </p>

One IMAGE in - or your own splat or 3D model, no fal needed. Four outputs, all from the exact camera angle you chose by hand:

| Output | Type | What it is | | --- | --- | --- | | image | IMAGE | The splat rendered from your viewport camera | | mask | MASK | Silhouette — 1 where the model covers a pixel | | depth | IMAGE | Depth of that same view; near surfaces are bright | | video | VIDEO | The camera move you recorded — RGB, mask, or depth — as clean constant-rate h264 |

The video output plugs straight into ComfyUI's Save Video node. The file is yours; what you do with it — camera reference for Seedance / Hailuo / Runway, previz, compositing — is up to you.

Two ways in: fal, or fully local

SplatCam is a camera rig for any 3D Gaussian splat - fal is the convenience path, not a requirement:

  • image input -> fal.ai TripoSplat builds the splat (~$0.05, cached so the same image never pays twice).

  • model_3d input -> fully local, fal is never called. Accepts:

    • ComfyUI's native SPLAT type (built-in TripoSplat Decode - the model is open source - or File3DToSplat);
    • 3DGS .ply / .splat files from any pipeline: gsplat training runs, Postshot, Luma exports;
    • any mesh: glb / gltf / obj / stl / mesh-ply files (via Load 3D) or ComfyUI's native MESH type from local image-to-3D models. Meshes are surface-sampled into Gaussians locally - geometry (and therefore depth and mask) stays exact, colour comes from the texture or vertex colours.

    For .spz/.ksplat convert with File3DToSplat; for .fbx/.usdz export as GLB first.

When both are connected, the 3D model wins. With only model_3d in use you don't need a fal key at all.

Install

Via Comfy Registry (ComfyUI-Manager):

SplatCam

Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/tahabahrami/ComfyUI-SplatCam
/path/to/ComfyUI/python -m pip install -r ComfyUI-SplatCam/requirements.txt

Restart ComfyUI, then hard-refresh the browser.

Configure the fal key

Splat generation runs on fal.ai TripoSplat (~$0.05 per image, billed by fal to your account). One-time setup:

python3 ComfyUI-SplatCam/scripts/configure.py

The key is stored in an owner-only (0600) file on your machine. It is never a widget, never in a workflow file, never in a log. FAL_KEY in the environment always takes precedence, so an operator-managed key can't be shadowed from the browser. --status, --link <path>, and --delete are also available.

Quick start

  1. Load Image → SplatCam. That's the whole graph.
  2. Queue once. TripoSplat builds a 262k-Gaussian splat and loads it into the node's viewport. Cached by image + settings — the same image never pays twice.
  3. Fly. Drag to orbit, scroll to zoom, shift-drag to pan. The mask and depth previews track the camera live.
  4. Record. Pick record_mode (use depth for camera references), press ● Record, fly your move, press ■ Stop.
  5. Queue again. The node emits image / mask / depth / video for your take. Connect video → Save Video and it's on disk.

Until you've framed a shot or recorded a take, downstream Save nodes are politely skipped with a message — SplatCam never writes black frames or empty files into your output folder.

Tutorial

▶ Watch the 35-second tutorial (mp4) — from install to saving your first depth video.

The camera-reference recipe

The strongest way to drive a video model's camera, learned the hard way:

  1. Set record_mode to depth and record your move. A depth take carries only geometry and camera motion — no colours, textures, or background a video model could copy by accident.
  2. Give the video model your source image as a reference image (not as a first frame — frame images and reference videos are competing modes on most providers, and the frame wins).
  3. In the prompt, say the reference video is only a camera-motion reference, and that all appearance comes from the reference image.
  4. Ask for the same duration you recorded, so the motion maps 1:1.

Sample prompt fragment:

Use the reference video ONLY as a camera-motion reference: match its orbit direction, speed, and timing exactly. Ignore its grey values and flat shading entirely — all appearance comes from the reference image. The subject stays still; only the camera moves.

Two provider gotchas worth knowing:

  • Some models enforce a minimum reference resolution (Seedance 2.0: 409,600 px). Keep capture_width × capture_height at 640×640 or larger.
  • Some models reject reference imagery containing realistic human faces, AI-generated or not. Stylised and non-human subjects pass.

Inputs

| Input | Default | Notes | | --- | --- | --- | | image | — | Optional. One IMAGE; fal builds the splat from it | | model_3d | — | Optional. A SPLAT, MESH, 3DGS .ply/.splat, or glb/gltf/obj/stl mesh — fully local, no fal | | num_gaussians | 262144 | TripoSplat target, and sampling density for mesh inputs | | num_inference_steps | 20 | 1–50 | | guidance_scale | 3.0 | 0–10 | | fal_seed | -1 | -1 lets fal choose. A new seed = new cache key = new paid generation | | capture_width / capture_height | 1024 | Size of the emitted stills and recordings | | record_fps | 24 | Recording + transcode frame rate | | record_mode | rgb | What a take captures: rgb, mask, or depth | | force_regenerate | false | Bypass the cache and pay fal again (advanced) | | viewport_state | — | Machine-managed; hidden |

Privacy & cost

  • Nothing leaves your machine unless you use the image path, in which case the source image alone is sent to fal.ai to generate the splat. The splat path is fully local. Recordings, captures, and splats always stay in your ComfyUI user directory.
  • You pay fal directly (~$0.05 per splat). SplatCam caches by image + settings and tells you in the log when a run reused the cache (fal was not called). Generation requests are submitted exactly once and never auto-retried, so a hiccup can't double-charge you.
  • The API key lives in an owner-only local file or FAL_KEY — never in workflows, widgets, or logs.

Design notes

  • Self-contained WebGL2 splat renderer — no CDN, no imports from ComfyUI's frontend bundle, so it survives frontend upgrades and works offline. One sorted pass produces colour, silhouette, and depth together via dual render targets.
  • Recordings are deterministic: locked to capture_width × capture_height (not the on-screen viewport), resampled to constant record_fps, with the real wall-clock duration of your take preserved even when the browser drops frames.
  • Strict provider hygiene: every URL from fal is checked against an allowlist before fetching; provider errors map to stable messages; polling is the only thing retried.

Development

node --test tests/frontend.test.mjs   # parser, camera, sort, state (no browser needed)
python3 -m pytest                     # payloads, storage, config, video, routes

126+ tests, zero-warning registry audit.

FAQ

Does it work without a fal account? Yes — connect the model_3d input instead of image: a locally built splat, a 3DGS file, or any glb/obj/stl mesh. fal is only used when you go through the image input.

Why two queues? ComfyUI executes on the server; your camera lives in the browser. The first queue builds the splat, you fly, the second queue emits your take. The splat cache makes the second queue free and instant.

GLB/mesh export? No — SplatCam is deliberately splat-native. If you need a mesh, use a mesh-generation node; if you need a camera, this is the node.

Licence

MIT — see LICENSE.