Nodes/ComfyUI-DragNUWA/InstantObjectMotionBrush
ComfyUI Node

InstantObjectMotionBrush

InstantObjectMotionBrush

By chaojie·Created 3 years ago·Updated 2 years ago· 411
InstantObjectMotionBrush
  • brush_mask
  • MotionBrush
model_length14
actionleft
speed5.00

InstantObjectMotionBrush is the object-motion sibling of InstantCameraMotionBrush, and it answers a different question. The camera version moves the whole frame; this one moves one thing - the thing you paint a mask over - while the background is supposed to stay put. Draw a mask on the car, pick right, set a speed, and you get a brush that pushes exactly that region forward.

It works by finding the bounding box of your mask and laying a grid of tracking points across it (again 10x10-ish). Each grid point only survives if it actually sits inside the mask - masked-out points are dropped. Then it computes the same synthetic per-frame paths as the camera node: translate for left/right/up/down, spread from the bbox center for zoomin/zoomout. Those paths become a dense MotionBrush flow tensor restricted to your mask region. The key difference from the camera node is that the displacement is scaled to the bounding-box size, so speed means something relative to the object, not the frame.

Inputs

  • model_length - default 14, match your checkpoint.
  • brush_mask - a MASK input, usually from a mask editor or an autosegment node. This is the whole game. A tight mask around the object gives clean regional motion; a mask that bleeds into the background drags the background along with it.
  • action - left, right, up, down, zoomin, zoomout.
  • speed - default 5.0 (note: higher default than the camera node, because it's scaled to the bbox). Start low and creep up.

Output

One MotionBrush. Feed it to DragNUWA Run MotionBrush with the matching source image. For a dolly effect you'll often combine it with InstantCameraMotionBrush via CompositeMotionBrush - subject moves one way, camera counters, the classic parallax look.

What actually bites people

The mask comes first. This node is not a segmenter. You need a real mask of the subject - paint it yourself or generate it (the repo's instant-motion workflow uses ResizeMask and mask-editing nodes feeding in). A bad mask is the #1 reason people declare this broken: the bbox math keys off the mask's bounding box, so a stray dot in the corner of the frame shifts the entire motion region and the model happily moves air.

Resolution consistency again. The brush's resolution is derived from the mask's shape, and it's applied at the checkpoint's dimension. Keep mask, brush, and model in the same coordinate space - resize the mask to match the checkpoint dimension before it arrives.

It's synthetic motion, not semantic. The node moves pixels within the mask; it has no idea what the pixels are. On a rigid object like a car that's perfect. On something deformable - a waving hand, a face - a uniform translate looks like the object is sliding on a track. Don't fight that; it's the tool's nature.

Same pack-level reality as the rest: SVD-era output quality, hardcoded cuda:0 at model load, heavy requirements. But for "make the subject drift while the world holds still," this is the fastest route in the pack - faster than drawing trajectories, and honestly the closest thing to a usable one-click experience DragNUWA ever got.

CategoryDragNUWA

Inputs (4)

NameTypeDefaultDescription
model_lengthINT14
brush_maskMASK
actionCOMBOleft6 options: left, right, up, down, zoomin, zoomout
speedFLOAT5.00

Outputs (1)

NameTypeDescription
MotionBrushMotionBrush