Nodes/Steerable Motion/Video Frame Extractor & Mask Generator πŸŽžοΈπŸ…’πŸ…œ
ComfyUI Node

Video Frame Extractor & Mask Generator πŸŽžοΈπŸ…’πŸ…œ

Build your own guidance video for VACE, frame by frame

By banodocoΒ·Created 3 years agoΒ·Updated 11 months agoΒ· 975
Video Frame Extractor & Mask Generator πŸŽžοΈπŸ…’πŸ…œ
  • input_video_frames
  • depth_video_frames
  • master_inpaint_mask
  • guidance_video_frames
  • guidance_frame_masks
β—„total_output_frames81β–Ί
β—„frame_selection_string0, 10:20β–Ί
β—„empty_frame_fill_level0.50β–Ί

VideoContinuationGenerator builds your guidance video for you - the tail of the last clip becomes the anchor, everything else is blank. But sometimes you don't want the pack's opinion on which frames matter. Sometimes you have a specific control video in mind, or you want to hand-pick frames for a depth or pose-driven VACE run. That's the job of Video Frame Extractor & Mask Generator πŸŽžοΈπŸ…’πŸ…œ: you tell it exactly which frames of an input video to place where, and it hands back a guidance video plus matching masks with zero ambiguity about what's "known".

What it does

You give it input_video_frames and a frame_selection_string - comma-separated integers and inclusive ranges like 0, 10:20 (default 0, 10:20). It pulls those frames out of the input, places each one at the same index in a fresh output timeline of total_output_frames, and marks them known (mask 0). Every frame you didn't select gets filled with the empty_frame_fill_level gray (0–1, default 0.5) and left as inpaint territory (mask 1). What comes out is a guide that says "these frames are real, the rest is yours to draw" - exactly the contract VACE wants.

Inputs worth knowing

  • total_output_frames must satisfy (frames - 1) divisible by 4 - the same Wan constraint as the pack's other continuation nodes, and it'll raise a hard error if you break it.
  • Optional depth_video_frames: give it a depth video and it fills any slot the selection string left empty, resizing the depth to match your input's resolution. Note the precedence: frame_selection_string wins every time, and depth frames stay mask 1 (inpaint) rather than being marked known - they guide the shape, not the pixels.
  • Optional master_inpaint_mask: if you plug one in, it defines the entire output mask, overriding everything derived from frame selection and depth. Handy when you know the exact region that should be redrawn; a trap if you just want to try it and forget it's there.

The outputs

guidance_video_frames (IMAGE) and guidance_frame_masks (MASK), both sized to your output length. Same destination as the continuation generator: into WanVideoVACEEncode, with the mask deciding what gets preserved. This is the node you reach for when a VACE workflow is being driven by a control sequence - depth, pose, a hand-authored video - rather than a plain tail-to-tail continuation, and it's the closest thing in the pack to a "build your own VACE input" tool.

Gotchas that will cost you a run

  • Ranges are inclusive. 10:15 selects six frames (10, 11, 12, 13, 14, 15), not five. Easy to mis-count when you're eyeballing a control video.
  • Out-of-bounds and negative indices are silently skipped with a log warning - your video won't fail, it'll just be missing frames you thought were there. Check the console.
  • The selection string outranks depth. If a slot is filled by the string, depth won't touch it. Wire it that way on purpose.

Install

The same one-liner as everything in this pack: ComfyUI Manager β†’ search "Steerable Motion", or git clone https://github.com/banodoco/steerable-motion into ComfyUI/custom_nodes/ and restart. The repo only requires matplotlib; the heavy lifting - Wan models, Kijai's WanVideoWrapper - lives in the workflows, and Manager will offer them when you load one. This node itself is pure tensor surgery: take frames, place frames, emit masks. Cheap on VRAM, occasionally expensive on patience.

CategorySteerable-Motion

Inputs (6)

NameTypeDefaultDescription
input_video_framesIMAGEInput video frames (IMAGE batch) to extract from.
total_output_framesINT811–10000Total number of frames for the output guidance video and masks. Must satisfy: (frames - 1) divisible by 4.
frame_selection_stringSTRING0, 10:20Comma-separated integers or ranges (e.g., 0, 5, 10:15, 20) of frames to extract from input video. Takes precedence over depth_frames.
empty_frame_fill_levelFLOAT0.500–1Grayscale level (0.0 black, 1.0 white) for frames not explicitly selected or filled by depth.
depth_video_framesoptIMAGEOptional depth frames (IMAGE batch). Placed if the slot is not already filled by frame_selection_string.
master_inpaint_maskoptMASKOptional master inpaint mask. If provided, it defines the entire output mask, overriding masks for selected/depth frames.

Outputs (2)

NameTypeDescription
guidance_video_framesIMAGEβ€”
guidance_frame_masksMASKβ€”