CameraPoseVideo

CameraPoseVideo is the reason this pack has two halves. Dust3rRun turns photos into a 3D scene; this node turns a sequence of frames into a camera trajectory - the exact CAMERACTRL_POSES format that CameraCtrl-style video control expects. So instead of hand-typing trajectory numbers or nudging preset motions, you shoot a short dolly move with your phone, run it through this node, and AnimateDiff follows the real move.

It's the niche sibling, but the payoff is genuinely fun: film a slow walk around a table, extract the poses, and have your animated generation reproduce that arc - zoom, tilt, parallax, the works.

How it works

For every consecutive pair of frames in your image batch, it runs dust3r's pairwise reconstruction (the lightweight PairViewer mode, not the full multi-view optimizer). It reads the relative camera pose between the two frames, then chains them cumulatively so frame N's pose ends up relative to frame 1 - that's the whole trajectory. Each pose is formatted into the CameraCtrl layout: a fixed per-frame prefix plus the flattened 3×4 rotation/translation matrix. Roughly 19 numbers per frame, one entry per frame in your sequence.

Because it works pair-by-pair, it's actually lighter on VRAM than Dust3rRun's full scene alignment - one ViT-Large pass per pair instead of an optimizer chewing on all frames at once.

Inputs and output

• model - a Dust3rModel from Dust3rLoader. Same checkpoint as the rest of the pack.
• images - an IMAGE batch of consecutive frames. This is the one that trips people up: it assumes temporal order. Feed it a video's extracted frames, not shuffled photos of a scene.
• image_size - 512 default, matching the 512 model.
• scenegraph_type - complete / swin / oneref. With two-frame pairs this matters less than on Dust3rRun; complete is fine.
• device - cuda default, cpu if you must.

The single output, CAMERACTRL_POSES, wires straight into a CameraCtrl node. The author of this pack also maintains ComfyUI-CameraCtrl-Wrapper, which accepts exactly this type - and AnimateDiff-Evolved's CameraCtrl block takes it too. On the other side of that node you'll need the CameraCtrl weights, the AnimateDiff v3 motion module, and its v3 adapter LoRA.

Installing it and the gotchas

Same install as the whole pack: ComfyUI Manager → search ComfyUI-dust3r, or git clone https://github.com/chaojie/ComfyUI-dust3r into ComfyUI/custom_nodes/, restart, and let the auto-install of dependencies (roma, trimesh, opencv-python, scipy, einops, pyglet) finish on first load. And yes - you still need to download the dust3r checkpoint into checkpoints/ yourself; the loader won't fetch it.

Real-world caveats: consecutive frames need to overlap enough for dust3r to register them - a camera jump mid-clip will produce a pose jump in the trajectory, and a dead-flat or featureless wall gives it nothing to lock onto. The result is a relative trajectory too, so don't expect metric scale; the CameraCtrl side just needs the motion shape. And remember the pack is CC BY-NC-SA 4.0 - fine for experiments and personal work, not for selling the output.