AniPortraitRun

AniPortraitRun is the node that actually makes the video. Feed it the pipe and a2m_model that AniPortraitLoader built, a reference portrait, a short driving video, and an audio file, and out comes a video of the portrait talking with the audio and moving its head the way the driving video moves. It's the entire "make the picture come alive" step in one box.

The flow is the classic AniPortrait recipe, which the source spells out plainly: detect the face in your reference image with mediapipe and draw its landmarks; run the audio through a wav2vec2 encoder and the audio2mesh model to predict a 3D head mesh that's lip-synced to the speech; then use mediapipe again to track head pose through every frame of the driving video, compute each frame's pose relative to the first, interpolate to 30fps, smooth it, mirror it, and tile it to the audio length. Those pose values drive the 3D mesh, which gets projected to 2D landmarks and drawn as pose-skeleton frames. Finally the Pose2VideoPipeline - the SD 1.5 UNet with its AnimateDiff-style motion module and pose guider from the loader - denoises those skeleton frames conditioned on your reference photo into a photorealistic sequence. That's a lot of moving parts, and the node hides all of them behind one run.

The inputs that matter

• pipe and a2m_model - straight from AniPortraitLoader. Non-negotiable; this node has no idea what to do without them.
• image - your reference portrait. The face has to be detectable by mediapipe; if it isn't, the node hard-asserts "No face detected." and dies.
• pose - an IMAGE batch of the driving video's frames. This is the "face reenactment" input: any video of someone moving their head will do, which is why the original paper and community grabbed random dancing/shuffling clips. Load it with a video loader like VHS_LoadVideo.
• audio_path - a WAV file. The default points at lyl.wav, a sample bundled inside the repo, so it works out of the box but you'll swap it immediately.
• video_length (16), steps (25), cfg (3.5), seed (1234) - the diffusion knobs. 16 is the motion module's native context window; you can push it but 16 is where this architecture is happiest.
• min_face_detection_confidence (0.5) - how sure mediapipe has to be before it trusts a face in the driving frames. If tracking drops frames, nudge it down.
• width/height (512 each) and weight_dtype (fp16) - resolution and precision. fp16 is fine; go fp32 if you hit NaNs or have the VRAM.

The output is a single IMAGE - but it's really a video wearing an IMAGE's clothes: shape (1, T, H, W, C), time axis second. Wire it into a video-combine node (the bundled wf.json uses VHS_VideoCombine) to get an actual mp4.

Installing and getting it to run

Same pack install as the loader - Manager or:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI-AniPortrait
pip install -r requirements.txt

And same model parade: the five ZJYang/AniPortrait weights, plus SD 1.5, sd-vae-ft-mse, the image_encoder folder, and wav2vec2-base-960h, all under ComfyUI/models/diffusers/. Full paths for every one of them live in the defaults on the loader. Keep the folder named ComfyUI-AniPortrait - the code resolves its own configs and the sample audio against that exact name.

The honest troubleshooting list

This is a 2024-era SD 1.5 pipeline, and the community said so at the time: the first impressions in the original AniPortrait threads were "looks choppy" compared to Alibaba's EMO, with "best ComfyUI talking-head yet" as the counter. By 2026 it's an old workhorse, not the state of the art - but it's fully local, weighs nothing compared to modern video models, and still does face reenactment plus audio lip-sync that plenty of newer ComfyUI packs made people install more stuff for.

Real failure modes, from the code and community reports: no face detected in the reference (use a cleaner, front-facing crop); the pose video not tracking (lower the confidence threshold or use a video with a clear face); OOM on 8 GB cards (512x512, 16 frames, fp16 is already the minimum - the loader alone eats most of that); and "module not installed" errors that trace back to skipping requirements.txt. If audio isn't syncing, check that your WAV is standard PCM - the pipeline extracts features with wav2vec2 and doesn't do exotic formats gracefully.