Load HelloMemeImage Pipeline

This node builds the actual diffusion pipeline that HelloMeme Image Pipeline samples with. Everything before it in the graph - face toolkits, landmarks, pose, expression - is just describing what the face is doing. This is the thing that decides who the face is, in the form of an SD 1.5 checkpoint, an optional LoRA, a VAE, and a HelloMeme version. It's the "model loading" step of the pack, and it's where most of the creative control hides.

One framing note up front: HelloMeme is built on SD 1.5. That's why the pack works with the whole legacy checkpoint/LoRA ecosystem, and also why its output carries that telltale SD look even at its best - a criticism the community leveled at the pack from day one. Pick your base accordingly.

The inputs that matter

Every input here is optional - the node has sane defaults and will happily pull the plain SD1.5 checkpoint from the cloud. In practice you'll touch these:

• checkpoint - pick a [preset] (RealisticVision, DreamShaper, a stack of anime and cartoon models) that auto-downloads from the author's HuggingFace, or select any SD 1.5 checkpoint you already have in ComfyUI/models/checkpoints. Realistic models look less like "SD 1.5" out of the box, which is worth a lot here.
• version - the HelloMeme algorithm version: v1 through v5c. v2 (the default) brought better expression consistency with the driving video, better third-party checkpoint compatibility, and lower VRAM. The later versions keep refining that. If a workflow specifies a version, keep it; otherwise v2 is the safe default.
• stylize - x1 (normal) or x2 (an extra super-resolution pass). x2 costs time and VRAM but sharpens the result noticeably.
• lora and lora_scale - an optional style/identity LoRA. The [preset] LoRAs (BabyFaceV1, MoreDetails, PixelPortrait…) come from the cloud; any LoRA in ComfyUI/models/loras works too. lora_scale defaults to 1.0.
• vae - same-as-checkpoint is fine; swap in a specific VAE if your base's baked one is mushy.
• deployment - huggingface or modelscope for where cloud weights download from.
• dtype - fp32 or fp16. If VRAM is tight, fp16 first, before dropping anything else.

Output: hm_image_pipeline (HMIMAGEPIPELINE), wired straight into HelloMeme Image Pipeline.

How it works

The loader builds a diffusers SD1.5 pipeline from the checkpoint, then injects the pack's trained "spatial knitting" modules (the contribution of the HelloMeme paper, arXiv 2410.22901) so the pipeline understands head pose and expression conditioning. It also swaps in your VAE and LoRA. This is why the first load of any version is slow - that's model assembly and download, not the generator struggling.

Installing it

Installs as part of the HelloMeme pack via ComfyUI Manager (hellomeme-api) or:

cd ComfyUI/custom_nodes
git clone https://github.com/HelloVision/ComfyUI_HelloMeme
# restart ComfyUI

The heavy diffusers stack (transformers, diffusers, accelerate, peft, onnxruntime…) comes with the pack install. Model files - checkpoints, LoRAs, HM modules - download automatically on first use, no manual placement needed.

Common issues

• First load of a new version is a big download. Pick your version once and stick with it across runs.
• "It looks too much like Stable Diffusion." That's the architecture talking. Switch to a realistic preset checkpoint and crank stylize to x2; the gap narrows but never fully closes.
• VRAM death on stylize x2. If sampling dies on the upscale pass, drop stylize to x1 and use an external upscaler - the pipeline passes a LATENT output precisely so you can do that.