Unimatch Optical Flow

This is the strange one in the pack. Every other node here makes a hint image - edges, depth, a pose skeleton. Unimatch makes optical flow: a per-pixel map of how things moved between one frame and the next. It's not conditioning a still image at all. It exists to feed DragNUWA, the drag-based motion-control video system, where the flow field tells the model which direction each part of the frame should travel.

If you're here from a ControlNet tutorial expecting another edge detector, this isn't that. It's a motion-estimation utility for a specific and fairly niche video pipeline.

What optical flow is and why it's here

Optical flow is the classic computer-vision idea of tracking motion: for every pixel in frame A, which way and how far did it go by frame B. Unimatch (the GMFlow family of models, weights from hr16/Unimatch) estimates that field from a batch of frames. DragNUWA then uses it as a control signal - you effectively hand it "here's the motion I want" and it animates toward it. That's the DragNUWA link the pack README points at.

Being honest about where this sits in 2026: DragNUWA had its moment, but drag/flow-based motion control never became the mainstream way people do video. The video-control conversation has moved on to other approaches entirely. So treat this node as a specialist tool for a specific DragNUWA workflow, not a general part of your kit.

The inputs and outputs

The required inputs:

• image - a batch of frames (this only makes sense on a sequence, not a single image).
• ckpt_name - which GMFlow checkpoint to use. The default gmflow-scale2-regrefine6-mixdata.pth is the most refined (and slowest); gmflow-scale2-mixdata.pth and gmflow-scale1-mixdata.pth are lighter, faster options.
• backward_flow (default false) - compute motion in reverse (frame B to A) instead of forward.
• bidirectional_flow (default false) - compute both directions.

Two outputs:

• OPTICAL_FLOW - the actual flow field, which is what you route into the DragNUWA node. This is the payload.
• PREVIEW_IMAGE (an IMAGE) - a colorized visualization of the flow so you can sanity-check it with your eyes. The direction/magnitude of motion shows up as color and intensity. This is for looking at, not for conditioning.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. The GMFlow checkpoint you select downloads from HuggingFace on first use.

Where people get burned

The number one confusion is wiring the OPTICAL_FLOW output somewhere it doesn't belong. It is not an image and not a ControlNet hint - it won't go into a ControlNet Apply node. It goes into a DragNUWA node that knows how to consume a flow field. If you don't have that downstream half of the pipeline, this node has nowhere useful to send its main output.

Second: a single frame gives you nothing. Optical flow is defined between frames, so you need at least two, and realistically a proper batch, for it to mean anything. Feed it one image and you're measuring motion against nothing.

Third, expectation-setting: this is a legacy-flavored corner of the pack. If your goal is modern controllable video, look at what the current video ecosystem uses rather than trying to build a DragNUWA workflow from scratch around this node.