Wan Pose 3D Refine — NLF (V2)

When you drive Wan 2.2 Animate from a video, the whole performance rides on a stick figure. Your frames go through a pose detector - in this pack, YOLO + ViTPose - which produces the OpenPose-18 keypoints that become the skeleton conditioning Wan uses to transfer motion onto your character. That works great on clean, well-lit footage, and falls apart the moment anything real happens: an arm crosses the body, a limb exits frame, the lighting shifts. The per-frame keypoints wobble, teleport, or just vanish, and because the sampler treats that skeleton as ground truth, the jitter you barely see on the skeleton overlay is the limb-flicker you absolutely see in the final animation.

Wan Pose 3D Refine - NLF (V2) is a fix applied at exactly the right place: between the detector and the sampler. It takes your already-detected pose_data plus the original images, runs them through NLF (Neural Localizer Fields) - a NeurIPS 2024 research model by István Sárándi and Gerard Pons-Moll that estimates 3D human body pose - and blends NLF's stable, occlusion-aware body into the shaky ViTPose keypoints. Same POSEDATA structure out as in, so it drops straight into the pose path you already have wired.

How it works

Per frame, it feeds the image to NLF's detect_smpl_batched and asks for SMPL-X body joints, then maps those canonical body indices onto the OpenPose-18 skeleton. A few things are genuinely well thought-out here, visible in the source rather than the marketing:

• An uncertainty gate. NLF reports per-joint uncertainty; joints that are outliers get skipped instead of dragging a limb off-screen.
• No clamping. Off-frame joints are simply left alone - clamping would pin a limb to the corner and draw a phantom arm.
• A one-Euro filter smooths the refined body joints over time when temporal_smoothing is on.
• Only the body joints (neck through ankles) are refined. Nose, eyes, ears keep their ViTPose values, so face/head jitter is not this node's job.

Honest caveats: it refines the largest detected person per frame, and NLF is a chunky academic model whose weights it auto-downloads (~470 MB) into ComfyUI/models/nlf/ on first run. The node's own docstring flags NLF as a research model - check the terms before building a commercial pipeline on it.

The inputs that matter

Most are already tuned; the two you'll actually touch:

• blend_strength (default 0.7) - how strongly NLF overrides ViTPose. 1.0 is full NLF. Drop it if the NLF projection ever fights the video's actual geometry.
• occlusion_fill (default on) - where ViTPose confidence is low, trust NLF's 3D projection instead. This is the one that rescues shots where an arm disappears behind the body.

pose_data (POSEDATA from this pack's PoseAndFaceDetectionV2) and images are required. temporal_smoothing with smoothing_min_cutoff / smoothing_beta tune the one-Euro filter; device defaults to auto and nlf_model lists any .torchscript in the models dir. Outputs: a refined pose_data you feed downstream as before, and an info STRING - read it, because on failure the node silently returns the original POSEDATA unchanged and tells you why in that string.

Installing it

Via ComfyUI Manager (search "WanAnimatePreprocessV2") or:

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-WanAnimatePreprocessV2

Restart ComfyUI. Since this is a whole preprocessing pack, the detection stage needs its own models first: YOLOv10m and a whole-body ViTPose ONNX file go in ComfyUI/models/detection/. The NLF model downloads itself on first run.

Where people get burned

• Node missing from the menu. Registration is guarded - if the module import fails, ComfyUI logs "WanPose3DRefineNLFV2 not registered" and the node simply doesn't appear. Check the console log.
• First run downloads ~470 MB from GitHub; behind a proxy that can fail. Drop the torchscript into ComfyUI/models/nlf/ manually and it'll pick it up.
• Expect rough edges. This node landed on main recently and has effectively zero community footprint yet - this is for people already fighting Wan Animate pose jitter, not a one-click solution. But if shaking skeletons have been eating your character-animation runs, it's the one I'd reach for.