Video To Depth

VideoToDepth runs a monocular depth estimator over every frame of a video and outputs a grayscale depth-map version of the clip - white close, black far, same convention every depth model in this ecosystem uses. It's the sibling node to VideoToCanny, and they exist for the same reason: to build the control-video input a Wan-Fun Control model needs. The README doesn't specify which depth network runs under the hood, so treat this as "a standard monocular depth estimator" rather than assuming it's a specific named model - what matters for using it is the output, not the internals.

Depth vs. Canny - the actual difference in practice

Both feed the same control_video slot on a Wan-Fun-Control sampler, so the choice between them is about how tightly you want to constrain the new video's shape. Canny gives you sharp, literal edges - great when you want the new content to follow the old content's exact contours. Depth gives you looser spatial and volumetric information - where things sit relative to the camera, not their precise outline - which tends to cooperate better when the subject's silhouette needs to change. Swap a fitted outfit for a flowing one and a Canny map will fight you, because the new shape has to occupy space the original edges never accounted for; a depth map is more forgiving there since it's encoding "this is roughly how far away things are," not "this is exactly where the lines are." If your control video's job is to lock geometry hard, reach for Canny. If it's to keep a scene's general layout and camera depth while giving the subject room to change, Depth is usually the better call.

The fields

Only two, and they're both about the input, not the method:

• input_video - the source clip, as an IMAGE batch (a stack of frames, the standard way ComfyUI represents video - no special video type involved).
• video_length (default 81, max 81, step 4) - how many frames to process. The step-of-4 matches the 4n+1 frame quantization the rest of this pack's samplers expect (Wan's VAE compresses time roughly 4x), so keep this aligned with whatever frame count your downstream generation is using or you'll end up with a depth video that's shorter than the clip it's meant to guide.

Output is images - a depth-map IMAGE batch. Wire it into the control_video input on WanFunV2VSampler, running a Wan-Fun-Control checkpoint.

Installing VideoX-Fun

ComfyUI Manager: search "VideoX-Fun" (its older name "CogVideoX-Fun" also finds it). Manual: cd ComfyUI/custom_nodes && git clone https://github.com/aigc-apps/VideoX-Fun, then restart.

No separate weights to download for this node itself - depth estimation is bundled into the pack. What you do need downloaded is the Wan-Fun-Control checkpoint you're planning to condition, dropped under ComfyUI/models/Fun_Models/ in the diffusers-folder layout this pack expects, not a loose safetensors file.

Common issues

Same trap as its sibling node: the output is a frame batch, not a video file, so you'll need a video-combine node downstream if you want to preview or save the depth map as an actual clip. Frame-count mismatches between this node's video_length and your sampler's video_length are the other recurring issue - get them out of sync and the tail end of your generation loses control conditioning partway through, which usually shows up as the motion drifting or losing structure in the last few seconds of an otherwise well-guided clip.

One thing worth knowing going in: depth estimation genuinely struggles with fast motion and motion blur, more than edge detection does, because it's inferring 3D structure frame-by-frame rather than just finding contrast boundaries. If your source footage is fast or shaky, expect noisier depth maps and correspondingly less stable control than you'd get on calmer footage - that's a property of monocular depth estimation generally, not something specific to this node.