[Inference.Core] Mask Optical Flow (DragNUWA)

If you landed here confused, that's a reasonable reaction - this isn't a preprocessor that reads an image, and it doesn't fit the usual "feed it a picture, get a control map back" pattern the rest of this pack follows. Mask Optical Flow reads an existing OPTICAL_FLOW field - the kind of data DragNUWA-style motion-control nodes produce - and masks it down to a specific region.

What DragNUWA is doing, briefly

DragNUWA lets you draw motion trajectories over an image or video and translates them into an optical flow field: per-pixel vectors describing how each part of the frame should move. That flow field then conditions a video generation pipeline, dragging pixels along the paths you specified - pan a background, spin an object, whatever trajectory you drew. It's genuinely useful and genuinely fiddly, and it's produced by a different node in your graph entirely (a DragNUWA trajectory node, not anything in this pack) - Mask Optical Flow doesn't create motion, it edits an existing motion field you already built elsewhere.

Why you'd mask a flow field

Left alone, a trajectory you draw affects the whole frame. Masking the flow field confines the controlled motion to just the region your mask covers - the subject moves the way you dragged it, the background stays put, instead of the whole image getting dragged along with your trajectory. That's the entire job of this node.

optical_flow is the flow field coming from your upstream DragNUWA-style node. mask is an ordinary ComfyUI MASK - drawn by hand, generated with SAM, or built however you'd normally produce a mask for any other purpose. The node zeroes or limits flow outside that mask.

There are two outputs. OPTICAL_FLOW is the masked flow field - feed it onward into whatever motion-conditioned sampler was going to consume the original field. PREVIEW_IMAGE renders the masked flow as an actual visible image, which matters more than it sounds like: optical flow data is otherwise invisible in the graph, so this is your only way to confirm the mask actually landed where you meant it to before committing to a full generation.

Who actually needs this node

This is narrow and workflow-specific. You'll only reach for it if you're already deep in a DragNUWA-style motion-control pipeline for video - if that phrase doesn't mean anything to you yet, you don't need this node right now, and that's fine. It's not a general-purpose tool.

Installing it

Bundled with the pack - ComfyUI Manager, search "ComfyUI-Inference-Core-Nodes", or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/LykosAI/ComfyUI-Inference-Core-Nodes

then install.py, or pip install -e . (the plain, non-accelerated form is fine here - this node does tensor math on an existing flow field, not neural-network inference, so the [cuda]/[rocm]/[directml] onnxruntime extras don't buy you anything for this specific node). Restart ComfyUI after.

Troubleshooting

The one real failure mode: a masked flow field that comes back all-zero or clearly wrong almost always means your mask and your optical_flow field aren't at the same resolution. Resize the mask to match the flow field's actual frame dimensions before feeding it in, rather than assuming the node will handle a mismatch for you.