Animal Pose and Detection

If you've ever driven a human character with Wan Animate, you know the trick: extract a DWPose skeleton from a reference video, feed the skeleton to the sampler, and the model animates your character to match it. This node is that pipeline, but for animals. Feed it your cat video, it hands back a 17-keypoint animal skeleton bundle that the rest of this pack turns into pose images for Wan Animate. It's the beating heart of ComfyUI-WanAnimalPreprocess, and it's honestly a small miracle it exists at all - animal pose estimation has been a wasteland next to the endless OpenPose/DWPose tooling for humans.

It's adapted from Kijai's ComfyUI-WanAnimatePreprocess, with the human models swapped for animal ones. The pack is a niche community fork, not a ComfyUI-registry staple, so you're on the bleeding edge - which is exactly why you're here, because nobody else is shipping animal pose preprocessing.

How it works

Two ONNX models, two stages. First a YOLOv8 detector finds the animal in each frame - it resizes to 640×640, runs detection, and keeps the highest-confidence bounding box, filtered to COCO animal classes (bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe). Then ViTPose, an animal-pose model, crops to that box and predicts 17 keypoints in AP10k/APT36k format: eyes, nose, neck, shoulders, elbows, front paws, hips, knees, back paws. Two models, one per frame, and the whole batch gets a progress bar.

The interesting part is the optional retargeting. If you pass a retarget_image, the node detects the reference animal's pose too, then warps the template video's skeleton onto the reference animal's proportions. That's what lets you take a running horse video and map it onto your own horse picture - the motion transfers, the body shape stays the reference's.

The inputs that matter

• model - the POSEMODEL bundle from ONNX Animal Detection Model Loader. You can't run this node without it.
• images - your video frames as an IMAGE batch. Load a video with VHS or the built-in loader and wire the frames here.
• width / height - default 832×480, matching a typical Wan output. These aren't the analysis resolution (that's fixed at 256×192 for ViTPose); they're the coordinate space the skeleton is expressed in. Match them to your WanAnimate generation size or your skeletons will be in the wrong frame.
• retarget_image (optional) - the reference animal you want the pose transferred onto.

What comes out

Three outputs, and you only need one most of the time:

• pose_data - the POSEDATA bundle (poses, reference pose, dataset). This is what you wire into Draw Animal ViTPose or the Prompt Helper.
• key_frame_body_points - a JSON string of body keypoint coordinates sampled from key frames. For people building custom logic or debugging.
• bboxes - the detected bounding boxes per frame.

If you're doing plain Wan Animate, the flow is loader → this node → Draw Animal ViTPose → sampler.

Installing

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-WanAnimalPreprocess.git
cd ComfyUI-WanAnimalPreprocess
pip install -r requirements.txt

You can also search "ComfyUI-WanAnimalPreprocess" in ComfyUI Manager. The real dependency weight is in the models: a ViTPose ONNX file (45–300 MB) and a YOLOv8 ONNX file, both dropped into ComfyUI/models/detection/ - the same folder ControlNet Aux reads.

Where people get burned

The dataset choice is a silent trap. AP10k and APT36k both use 17 keypoints in the same layout, so a mismatched dropdown doesn't error - it just produces nonsense keypoint positions. Match the loader's dataset to whatever your ViTPose model was trained on.

Also, this is the slow part of any workflow. YOLO plus ViTPose per frame on CPU is a slideshow; use the CUDA provider. And if detection fails on a frame, the node falls back to the full frame as the bounding box - useful (it degrades gracefully) but it means one badly-framed clip gives you a jittery skeleton, because the fallback box is huge while the detected ones are tight. Check the bboxes output if your skeletons are wildly unstable between frames.