Nodes/ComfyUI-UniAnimate-W/Align & Generate poses for UniAnimate
ComfyUI Node

Align & Generate poses for UniAnimate

Where every UniAnimate workflow actually starts

By Isi-dev·Created 2 years ago·Updated 11 months ago· 187
Align & Generate poses for UniAnimate
  • reference_image
  • video
  • ref_pose
  • pose_seq

The most searched node in this pack is also the most boring, and that's exactly why you need it. Align & Generate poses for UniAnimate doesn't animate anything. It's the door you walk through: it turns your reference photo and your driving video into the two pose images every UniAnimate and Animate-X animation node expects as input. Miss this node and the rest of the pack is useless; hit it right and the rest of the graph just works.

What it does

UniAnimate is a human image animation method - you hand it a photo of a person and a video of someone else moving, and it makes your person copy the motion. It doesn't understand raw video, though. It understands DWPose skeletons: the stick-figure-with-face-and-hands maps that ControlNet users know well (controlnet.md has the full lowdown on DWPose's better hand detection). This node is the DWPose factory for the pack.

It takes two inputs and does three jobs:

  1. reference_image - your photo of the person. It extracts a pose from this.
  2. video - the driving clip, loaded as a batch of frames. It extracts a pose for every frame.

Then, critically, it aligns the driving pose sequence to your reference person. It scales the whole pose sequence up or down using the first frame of the driving video as the scale coefficient, so your character's body proportions roughly match the motion source. That alignment is the whole point of this node - the Generate dwpose sibling (Gen_align_pose2) skips it, and you can feel the difference.

The outputs and where they go

  • ref_pose - the single aligned pose of your reference image. Wire this into the ref_pose input of Animate image with UniAnimate or Animate image with Animate_X.
  • pose_seq - the aligned per-frame pose batch. Wire this into the pose_sequence input of the same nodes.

That's the whole wiring pattern: this node feeds the two Animate image with... nodes, and nothing else. The Long and v2 variants of the pack generate their own poses internally, so they don't need it.

Installing it

The pack is Isi-dev/ComfyUI-UniAnimate-W. In ComfyUI Manager, search "UniAnimate Nodes for ComfyUI", or clone it:

cd ComfyUI/custom_nodes
git clone https://github.com/Isi-dev/ComfyUI-UniAnimate-W

Then restart ComfyUI. The heavy part isn't the install - it's the models. This node specifically needs the two DWPose ONNX files (dw-ll_ucoco_384.onnx and yolox_l.onnx) in checkpoints/, plus the rest of the ~14GB stack if you want the animation nodes to run at all. The README's python modeldownloader.py (ModelScope) pulls everything; see the pack docs for the manual HuggingFace route. Python 3.9+, xformers, and onnxruntime are all required.

Gotchas

The README is blunt about the failure mode here: if the driving video can't be read by the DWPose detector, this node throws cannot convert float infinity to integer. It's the pack's most common complaint and it always traces back to a bad input, not a broken install.

Quality tip from the author: make sure the first frame of the driving video shows the full body including face, hands, and feet - it's the frame used to compute the scale alignment, and a cropped first frame degrades the whole result. And one genuinely neat trick: generate a video, then feed its last frame back as the reference with useFirstFrame enabled to extend the animation seamlessly.

Categoryimage

Inputs (2)

NameTypeDefaultDescription
reference_imageIMAGE
videoIMAGE

Outputs (2)

NameTypeDescription
ref_poseIMAGE
pose_seqIMAGE