Wan Video Motion To Flowmap (QQ)

Motion-based control for Wan is a hot mess of preprocessing steps. You usually have to estimate optical flow somewhere outside the graph - or hand it a flowmap video you made in another tool - before you can drive a "wan move"-style workflow. WanVideoMotionToFlowmap is this pack's attempt to do that estimation inside ComfyUI: it watches the motion between frames of any video you give it and emits an RGB flowmap, with the direction packed into red and green and the magnitude into blue.

It's one of the genuinely experimental nodes in siraxe's ComfyUI-WanVideoWrapper_QQ pack (now ComfyUI-SA-Nodes-QQ), which builds on Kijai's ComfyUI-WanVideoWrapper. The README frames the pack's "wan move" example as a mix of ATI-style and control 2.1 ideas, and this node is squarely in that neighborhood - it exists to feed motion signals to control workflows that want them as images. It's also not alone: the same pack ships a WanVideoFlowmapDistortion sibling that does the reverse, warping a video through a flowmap.

How it works

The node compares consecutive frames and produces, per frame, a three-channel map: R = horizontal displacement, G = vertical displacement, B = magnitude. Values are normalized so that 0.5 means "no movement" - a completely static clip comes out a flat mid-gray, which is a useful sanity check the first time you run it. The first frame copies the second frame's flow, so the output has the same frame count as the input.

The mode dropdown is where the real choices live, and the tradeoffs are honest:

• gradient_flow (default) - GPU, fast, a simplified Lucas-Kanade-style brightness-constancy estimate in pure PyTorch. Good enough for most purposes.
• frame_diff - the cheapest option; frame difference plus Sobel gradients. Fast, crude, fine for rough motion.
• centroid_track - tracks bright regions and builds a flow field around them. Reach for it when your video has a distinct bright object (an orb, a light source) doing the moving.
• optical_flow - proper OpenCV Farneback. The most accurate, but it needs OpenCV installed and silently falls back to gradient_flow if cv2 is missing - so don't assume you got Farneback quality just because you selected it.

Then three tuning knobs: strength multiplies the flow (default 1, up to 10), smoothing blends each frame's flow with the previous one to suppress jitter, and threshold matters mostly in centroid mode, controlling how bright a region has to be to get tracked.

The inputs that matter

• video - any image sequence; motion is estimated between consecutive frames.
• mode - the algorithm (above). This is the one you'll change.
• strength / smoothing / threshold - leave defaults alone until the flowmap looks wrong.

Output is a single flowmap image, same frame count as the input, ready to feed a flowmap-consuming control node downstream.

Installing it

Same as the rest of the pack - ComfyUI Manager, search "ComfyUI-SA-Nodes-QQ", or:

cd ComfyUI/custom_nodes
git clone https://github.com/siraxe/ComfyUI-WanVideoWrapper_QQ.git

Restart after cloning. No declared pip dependencies, but if you want the optical_flow mode to actually be optical flow, make sure OpenCV is in your Python environment (pip install opencv-python).

Where people get burned

The optical_flow silent fallback is the big one - pick it, get gradient_flow, and never know. Check your console output for the "OpenCV not available" warning if Farneback matters to you. Second, flowmaps from this node are estimates, not ground truth: frame_diff and gradient_flow will happily report spurious motion on noisy or high-contrast footage, so raise smoothing or swap modes before you blame the downstream node. And don't expect the output to look "natural" - a flowmap is data encoded as color, and flat mid-gray is the neutral you're aiming for, not a failure.