VVL VGGT Video Camera Estimator

Point this node at a video and it'll tell you exactly where the camera was for every frame - position, orientation, and the lens math to go with it. That's a genuinely useful trick for a ComfyUI node, because it's the bridge between the real world and the synthetic one: a handheld shot becomes a camera path you can read, compare against, or feed into something camera-aware. This is the actual workhorse of the VVL Video Camera Advanced pack; the loader node just feeds it a model.

The model behind it is VGGT-1B, Meta's Visual Geometry Grounded Transformer. VGGT is a feed-forward 3D-vision model - no per-scene optimization, no Structure-from-Motion solver running for minutes. You hand it a few frames and it predicts camera poses, intrinsics, and depth in a single forward pass. That's the whole appeal: it's fast and it's robust on messy real-world footage. Here's what happens under the hood when you hit run:

1. OpenCV opens the video and grabs frames every frame_interval frames, up to max_frames total.
2. Those frames get written to a temp folder as PNGs, then run through VGGT's official preprocessing.
3. The model predicts a pose_enc tensor, which gets decoded into per-frame intrinsic (3x3 K) and extrinsic (3x4 [R|t]) matrices, and camera positions via -Rᵀt.
4. Everything is serialized to JSON, and a matplotlib 3D plot of the camera path is rendered to an image.

The inputs that matter

• vggt_model - wire this from VVL VGGT Model Loader. There's no model-less mode; you need the loader in the graph.
• video - a video object from something like LoadVideo (the VIDEO type), or a plain file path string.
• video_path - a fallback path the node falls back to when video comes in empty. Honestly, this is the one I reach for first; it sidesteps any uncertainty about what object type your video source hands you.
• frame_interval (default 5) and max_frames (default 60) - these two control cost. Lower interval = denser sampling = slower. Keep an eye on max_frames; with interval 1 and a long clip you can blow past what you actually need.

What comes out

Three outputs:

• intrinsics_json - per-view focal-length/intrinsic matrices as a JSON string.
• poses_json - per-view extrinsic matrices plus camera position, also JSON.
• trajectory_preview - an IMAGE showing the camera path in 3D (with start/end markers and direction arrows). Wire it to SaveImage to actually keep it.

The JSON is the real payload here - that's the camera track you'd hand to a downstream 3D-aware tool. Be honest with yourself about the ecosystem, though: ComfyUI doesn't yet have a rich family of nodes that consume VGGT pose JSON, so today the practical consumers are "you, reading it" and the preview plot. That will change as camera-conditioned video generation matures; for now this is mostly a great way to see your camera path and get it out of the node graph in a parseable form.

Installing and running it

Install the pack via ComfyUI Manager (search "ComfyUI VVL VideoCamera Advanced") or clone it:

cd ComfyUI/custom_nodes
git clone https://github.com/wTechArtist/ComfyUI_VVL_VideoCamera_Advanced

First run downloads facebook/VGGT-1B (~4.7GB) from HuggingFace, so budget that. You want a CUDA GPU here - the node uses bf16 on Ampere+ cards (RTX 30-series and newer), fp16 otherwise, and plain fp32 on CPU. CPU technically runs it. "Technically" is doing a lot of work in that sentence; it's slow enough that you'll want the GPU.

Where people get burned

• The "Insufficient Camera Data" placeholder. The preview falls back to a gray "need at least 2 frames" image when there aren't enough poses - raise max_frames or lower frame_interval.
• File-not-found errors. The node resolves video paths by probing common attribute names on the video object, which is fragile. When in doubt, put the path directly in video_path and leave video empty.
• Missing cv2 or matplotlib. They're used but not listed in the pack's requirements. If you hit ModuleNotFoundError, install them yourself: pip install opencv-python-headless matplotlib.
• The torch pin. The pack's requirements pin an older torch; let ComfyUI Manager skip downgrading your environment if it asks.