ONNX Animal Detection Model Loader

Every workflow in ComfyUI-WanAnimalPreprocess starts at this node, because there's nothing in the pack that can run without the model bundle it produces. It loads two ONNX models - a YOLOv8 detector and a ViTPose animal-pose estimator - and bundles them with your dataset choice into a single POSEMODEL you feed to Animal Pose and Detection. It does nothing clever, and that's fine. It's the load-bearing boring node, and it's where most of your mistakes happen.

A quick translation, because the model names look like alphabet soup: ViTPose is the vision-transformer pose estimator that predicts the 17 animal keypoints (eyes, nose, neck, shoulders, elbows, paws, hips, knees). YOLOv8 is what finds the animal in the first place so ViTPose has something to keypoint. The code filters YOLO's output to COCO animal classes and keeps the highest-confidence box per frame, so a random human in the shot won't hijack your skeleton.

The inputs

• vitpose_model - dropdown of .onnx files found in ComfyUI/models/detection/. Pick your animal pose model.
• yolo_model - same dropdown folder, your detector. yolov8m.onnx or yolov8l.onnx are the README's recommendation for speed/accuracy balance.
• dataset - ap10k or apt36k. This is the one that gets people. Both datasets use the identical 17-keypoint skeleton, so nothing errors when you mismatch - you just get silently wrong keypoints. AP10k is the default and covers common animals (cat, dog, horse) across 23 families; APT36k has 30 species and more training data. Pick the one your ViTPose model was trained on, not the one you wish it was trained on.
• onnx_device - CUDAExecutionProvider (default) or CPUExecutionProvider. Keep CUDA unless you have no GPU.

Where the models come from

This is the real setup cost. The node loads files from ComfyUI/models/detection/, and nothing is bundled - you download both yourself:

# ViTPose ONNX from JunkyByte/easy_ViTPose on HuggingFace
# vitpose-{s,b,l,h}-{ap10k,apt36k}.onnx  (45MB / 90MB / 150MB / 300MB)
wget -P ComfyUI/models/detection/ \
  https://huggingface.co/JunkyByte/easy_ViTPose/resolve/main/vitpose-b-ap10k.onnx

# YOLOv8 ONNX - grab the .pt from Ultralytics and export, or download a pre-made export
wget -P ComfyUI/models/detection/ \
  https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt
yolo export model=yolov8m.pt format=onnx   # then move yolov8m.onnx into models/detection/

The vitpose-b-ap10k.onnx at ~90 MB is the sweet spot - noticeably better than small, not as glacial as huge. The .pth files the easy_ViTPose repo lists are training weights; the node wants the ONNX export, which is what the loader's dropdown shows.

Output

One output: model (POSEMODEL), the ViTPose + YOLO + dataset bundle. Wire it into the model input on Animal Pose and Detection or Animal Pose Detection OneToAll Animation. That's the entire job.

Install and gotchas

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-WanAnimalPreprocess.git
pip install -r requirements.txt

Search "ComfyUI-WanAnimalPreprocess" in ComfyUI Manager and you're done.

The silent-CPU trap: the dropdown says CUDAExecutionProvider, but if you only have onnxruntime (CPU build) installed, onnxruntime can't honor that provider and the session falls back to CPU - sometimes with only a warning log line. The pyproject dependency is onnxruntime-gpu>=1.16, so make sure that's what's installed, and match your CUDA toolkit version, or your "GPU" run will quietly be CPU. Worth a quick check of the console on your first run.