HelloMeme Video Pipeline

This is the node people install the pack for. HelloMeme Video Pipeline takes one reference photo, a driving video (any clip with a face doing something), and returns a video of the reference face performing the drive's motions - speaking, blinking, turning. It's the same reenactment idea as HelloMeme Image Pipeline, except the drive is a sequence of frames and the output is a sequence of frames with temporal consistency between them. The video-generation workflow (video_generation.json in the pack) is built around it.

Position it honestly: this was the pack's claim to fame at launch (accurate expression following) up against LivePortrait, the lighter and faster community standard. For driving a photo with a precise, subtle facial performance, HelloMeme's conditioning is strong. But it's SD 1.5 underneath, so your output carries that SD texture no matter how good the tracking is, and video multiplies both the VRAM bill and the render time.

How it works

It does the same heatmap-and-expression conditioning as the image pipeline, then runs the sampling over the driving clip in chunks, stitching them with overlap so frames don't jump when one chunk ends and the next begins. That chunk seam is why the video version has an extra dial the image one doesn't - patch_overlap. The reference pose/expression keep identity anchored, the drive pose/expression supply the motion.

The inputs that matter

• hm_video_pipeline - from Load HelloMemeVideo Pipeline (the video loader, not the image one).
• ref_head_pose / ref_expression - GetHeadPose / GetExpression run on the single reference image.
• drive_head_pose / drive_expression - the same nodes run on the whole driving clip (every frame).
• patch_overlap (default 4, range 0–5) - the overlap between generated chunks. Raise it if you see flicker or a "cut" between chunk boundaries; lower it to save time.
• trans_ratio (default 0) - how much head translation transfers, same as the image node; keep low unless you want the head to travel.
• steps, seed, guidance_scale (2.0), prompt / negative_prompt, gpu_id - same behavior as the image pipeline. Empty prompt = the pack's default prompt.

Outputs: IMAGE (all frames of the result, as a batch - wire it into a video-writer node like VHS to save a file) and LATENT (the frame latents).

Installing it

Part of the HelloMeme pack. ComfyUI Manager keyword hellomeme-api, or:

cd ComfyUI/custom_nodes
git clone https://github.com/HelloVision/ComfyUI_HelloMeme
# restart ComfyUI

The usual heavy deps (transformers, diffusers, accelerate, onnxruntime, opencv-python…) install with the pack; models auto-download on first use.

Common issues

• Flicker at chunk boundaries. Raise patch_overlap. That dial exists for exactly this.
• VRAM. Video is the memory hog of this pack. The README's own numbers are honest about it - a lighter non-AnimateDiff path was tested to run under 12GB, but the full video pipeline on a long clip will push a mid-range card. Use v2 or later, keep clips short, avoid stylize x2 on the loader.
• Drive clip with a lost face. If any driving frame lacks a detectable face, the landmark node upstream aborts - so your drive must keep the face on screen the whole time. Trim before you queue.
• Morphing on the face. Same story as the image pipeline: a clean front-on drive and a well-cropped reference fix more than any sampler setting.