Pose Generate Video 🎥AniPortrait

This is the node the whole pack is named around. AniPortrait_Pose_Gen_Video ("Pose Generate Video🎥AniPortrait") takes a reference photo and a sequence of face-landmark skeleton frames, and renders the photo performing that motion. Feed it the pose frames from AniPortrait_Video_Gen_Pose and you've got the pack's flagship "raw video to pose video with reference image" mode - a self-driven portrait animation that copies someone else's face movement onto your subject. It's the SD 1.5-era answer to "make this person do that person's talking."

How it works

The renderer is the full AniPortrait diffusion stack: the reference image gets encoded by a CLIP image encoder into a reference UNet, while the pose skeleton frames are fed through a pose guider into a 3D denoising UNet - the SD 1.5 UNet with a temporal motion module injected, the same AnimateDiff lineage - and a DDIM scheduler steps out the frames. Two things keep the face consistent across the clip: the reference UNet's identity injection, and the pose guider forcing the geometry. With accelerate on (default), it renders every Nth frame (via fi_step) and then film_net_fp16.pt frame-interpolates the gaps - that's the "speed up generation" trick in the README's changelog, and it's worth leaving on.

The honest tradeoff: the whole stack - SD 1.5, VAE, image encoder, motion module, pose guider, two more UNet weights - is loaded from disk on every run. There's no model caching, so each generation starts with a long reload. This is a "set it running and go make tea" node, not an interactive one.

Inputs that matter

• ref_image (IMAGE) - your subject's photo. Load Image → here. If no face is found it asserts with "Can not detect a face in the reference image," so pick a clear, centered shot.
• pose_images (IMAGE) - the landmark skeleton frames, normally from AniPortrait_Video_Gen_Pose's pose_images output.
• frame_count (INT) - this one's forced to be an input connection, so don't try to type a number. Wire it from VHS Video Info's frame-count output (that's exactly what the bundled pose2video_workflow.json does) so the render length matches your pose video.
• height / width - 512×512 defaults. Square.
• seed / cfg / steps - 42 / 3.5 / 25 defaults. Classic SD 1.5 CFG behavior; low-to-mid values are the sweet spot, and the same seed gives you a reproducible take.
• weight_dtype - fp16 or fp32; fp16 unless you hit NaN/black-frame trouble, then fp32.
• accelerate / fi_step - leave on (3) for the interpolation speedup.

Everything else is the pre-filled model-path enum list (vae_path, model, motion_module_path, image_encoder_path, denoising_unet_path, reference_unet_path, pose_guider_path). They're read from the pack's config and point at pretrained_model/ inside the pack folder - don't change them, just make sure the files are there.

Output

It returns images (IMAGE) and is marked as an output node, but as with everything in this pack, you still want to finish with VHS Video Combine to actually save an .mp4 (the workflow uses a 25fps h264 combine). The rendered frames are also what you'd feed into a video upscaler if you want to take the 512×512 output somewhere bigger.

Install - same heavy pack

cd ComfyUI/custom_nodes
git clone https://github.com/frankchieng/ComfyUI_Aniportrait
cd ComfyUI_Aniportrait && pip install -r requirements.txt

or search "ComfyUI_Aniportrait" in ComfyUI Manager and restart. Then populate pretrained_model/ in the pack directory with SD 1.5 (runwayml/stable-diffusion-v1-5), sd-vae-ft-mse, the image_encoder from lambdalabs/sd-image-variations-diffusers, and the AniPortrait weights from ZJYang/AniPortrait on HuggingFace: denoising_unet.pth, reference_unet.pth, pose_guider.pth, motion_module.pth, and film_net_fp16.pt (you can skip the audio2mesh/audio2pose pair unless you're also doing audio-driven work). The exact folder layout matters - the README shows the tree, and getting wav2vec2-style paths wrong is the classic failure.

Common issues

• Missing/weird output usually traces back to a non-square source, a reference photo with no clear face, or frame_count left unconnected.
• Black frames / NaNs: try weight_dtype = fp32.
• Old diffusers breaks on load (the PositionNet/GLIGENTextBoundingboxProjection rename); the pack pins diffusers==0.26.2.
• VRAM: this is a multi-model stack. On 8GB cards you'll want fp16 and short clips; expect OOM if you push length and resolution together.

If you're weighing whether this is the right tool in 2026, the honest answer is that WanAnimate and LTX-2 moved past the whole AniPortrait generation. But on an SD 1.5-era install, with the models already downloaded, this node still reliably does the one thing it's famous for.