DragNUWA Run

This is the heart of the pack. DragNUWA Run takes a starting image, a set of drag trajectories, and produces a batch of frames where the image actually moves along your paths. If you've ever used DragGAN-style editing and wished the result were a video instead of a still, this is that idea made real: you scribble where things should go, and the model figures out the motion.

Under the hood it's Stable Video Diffusion conditioned on a "drag map" - a per-pixel flow field built from your tracking points, interpolated across every frame of the clip. The node takes your sparse points, spline-interpolates them into a smooth trajectory per drag handle, and writes the displacement into a (frames, height, width, 2) tensor that the UNet treats as motion guidance. One frame of SVD at a time, 14 frames total, at 4 fps by default.

The inputs that matter

• image - your starting frame, as a normal IMAGE. It doesn't have to already match the model's dimension; the node center-crops it to whatever resolution your Load CheckPoint DragNUWA is set to. Crop, not squish - expect edges to get cut.
• tracking_points - a JSON string, which is the field that trips everyone up. Format is nested lists: [[[x0,y0],[x1,y1],...], ...], where each inner list is one drag handle's path across frames, and each [x,y] is a point on that path in order. You don't have to hand-write this: the author ships a web tool at tools/draw.html (open it in a browser, load your image, draw the trajectories, copy the generated textarea). Points are in model-pixel coordinates, so on a 576x320 canvas you're working with x up to 576 and y up to 320.
• motion_bucket_id - SVD's motion strength dial, 1-100, default 4. Lower = calmer, higher = more aggressive motion. It's an aesthetic knob more than a precision control; expect to play with it per image.
• use_optical_flow + directory - the escape hatch. Flip use_optical_flow on and it loads Middlebury .flo flow files from the given directory and uses those as the motion field instead of your tracking points. Leave it off unless you're deliberately doing the optical-flow workflow.

inference_batch_size exists in the schema but is locked to 1 (min = max = 1). Don't touch it.

The output

You get a single IMAGE - a batch of frames, 14 by default, which ComfyUI shows as an animated preview. To save it as an actual video file you'll want VideoHelperSuite's VHS_VideoCombine on the end; that's exactly how every example workflow in the repo is wired.

Common issues

The motion is weak or nothing moves. Bump motion_bucket_id, and make sure your trajectories have clear displacement - a handle that only moves a couple of pixels across 14 frames reads as "hold still." The drag tool lets you draw generously; use it.

Wrong resolution crop. Your image gets center-cropped to the model dimension, so a tall portrait image on a 576x320 model loses its top and bottom. Match the image aspect ratio to the chosen dimension before it hits this node.

It's slow. This is a 2024-era SVD fine-tune running locally. Even on a strong GPU you're waiting a while per 14-frame clip. Set expectations, queue a few variants, go make coffee.

Also remember this pack loads on hardcoded cuda:0 and drags in a heavy requirements.txt (pinned opencv 4.6.0.66, kornia 0.6.9, the whole SVD stack). If your environment suddenly misbehaves after installing, that's the likely suspect.