Load MotionBrush From Optical Flow

Load MotionBrush From Optical Flow is the pack's "copy motion from real video" node. Instead of hand-drawing trajectories or synthesizing camera moves, you feed it an optical flow tensor - motion vectors computed from an actual video - and it converts that into a MotionBrush that makes your starting image move like the source footage did. It's the mechanism behind the optical-flow workflow in the README, and it's a genuinely clever idea: real-world motion, extracted and re-applied to a still.

The source of the flow matters. The pack's optical-flow workflow gets it from Unimatch_OptFlowPreprocessor (Fannovol16's Unimatch pack, which the README credits), which computes dense per-pixel motion between frames of a video you load. This node takes that OPTICAL_FLOW tensor and - frame by frame, up to model_length - 1 frames - bilinearly resizes each flow field to the model's resolution and stacks them into the (frames, height, width, 2) brush format. The model then treats that copied flow as motion guidance for your image.

Inputs

• model - the DragNUWA object, needed to know the target resolution and model_length.
• optical_flow - an OPTICAL_FLOW-typed tensor from a flow preprocessor. This is the only input that produces this type, which is why the optical-flow workflow is its own thing rather than a variation of the basic one.

Output

A MotionBrush, sized to the model. Standard route into DragNUWA Run MotionBrush or a composite.

Why you'd reach for it - and why it's fiddly

The payoff is motion that looks physical - water, cloth, camera shake sampled from reality beats anything synthetic. The cost is the pipeline: you need a real video, a flow preprocessor installed and working (Unimatch isn't this pack's dependency, it's yours to add), and the video's motion has to be extractable at a frame count that lines up with model_length. The README's optical-flow workflow exists to hold your hand, but it's the most dependency-heavy path in the pack.

Expect friction: Unimatch is another install with its own weights, and flow quality depends on your source footage having clean, visible motion - flat or static footage gives you a brush with nothing in it. Also keep the usual discipline: source flow and model resolution must agree, and the flow's frame count should cover your model_length.

If the whole flow-preprocessing chain sounds like a project, it is. Start with InstantCameraMotionBrush for easy wins and treat this node as the advanced move for when you specifically want motion that traces reality. Same pack-wide caveats - SVD-era output, hardcoded cuda:0 at model load, heavy requirements - and one more: you're installing and debugging two packs' worth of dependencies for this one workflow.