Hallo Node

Give Hallo Node one portrait and one audio file, and it returns a video of that face talking, lips synced to the voice. No API, no key, no cloud - it's a ComfyUI wrapper around Fudan University's Hallo model, which was one of the first decent open-source audio-driven talking-head models back in 2024. It's the thing people reach for when they need "D-ID, but local."

Fair warning up front: this node is slow. It's not a slick native implementation; it shells out to Hallo's own Python inference script as a subprocess and waits. On a 4090 people report roughly a minute of rendering per second of video, and on a 12GB 3060 a 10-second clip can take the better part of an hour. Minimum is about 8GB VRAM, with ~9.8GB actually in use. If you just want a quick avatar talking, EchoMimic or the commercial tools are friendlier. You reach for this when you want Hallo specifically, or when you want the whole thing inside your ComfyUI graph.

How it works

Under the hood it's an SD 1.5-based pipeline with an AnimateDiff-style temporal motion module bolted on - check mm_sd_v15_v2.ckpt in the config and you'll recognize the architecture immediately. The driving audio gets split into vocals (via a Kim_Vocal_2.onnx separator, so background music won't drag the lips around), encoded with a Wav2Vec2 model, and injected into a 3D denoising UNet that also takes a face-locator crop of your source image as a reference. 40 DDIM steps later you get 512×512 frames at 25 fps.

The node itself is a thin harness: it writes a temp config, saves your image and audio to disk, runs scripts/inference.py in a subprocess, then reads the resulting MP4 back with OpenCV into a normal IMAGE batch. First run auto-downloads the entire pretrained bundle from fudan-generative-ai/hallo on Hugging Face into ComfyUI/models/hallo - several gigabytes, so give it a minute.

The inputs that matter

You mostly set four things:

• source_image - your portrait. The model crops to the face, so a clean front-facing headshot works best.
• driving_audio - an AUDIO socket, wired straight from ComfyUI's built-in Load Audio node (the pack added support for it mid-2024).
• lip_weight / face_weight / pose_weight - how hard the audio drives lip motion, facial expression, and head pose respectively. All default to 1.0. Lower lip_weight if the mouth looks too elastic; bump pose_weight if the head is too stiff.
• face_expand_ratio (default 1.2) - how much the animated region grows beyond the detected face box. Keep it near default.

Outputs are images (a batch of frames you can feed to a VAE decode or a video writer), count (number of frames), and frame_rate - which the node reports as a flat 25.

Installing it

ComfyUI Manager is the easy path: search for "hallo" and install. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/hay86/ComfyUI_Hallo
cd ComfyUI_Hallo
sudo apt install ffmpeg
pip install -r requirements.txt

Then restart ComfyUI. Note this requirements.txt is heavy - onnxruntime-gpu, audio-separator, insightface, mediapipe, decord, av, moviepy, bitsandbytes - and it drags in torchaudio and xformers, so it's not a clean fit for an existing well-behaved environment. Back up your venv or use a separate one if that scares you.

Where people get burned

• It looks uncanny. Hallo v1 is a research model; mouth shape and expression drift under stress. A common community trick is to run Hallo for the base lipsync and pass the result through LivePortrait to re-animate the face for more natural motion.
• Slow first run and slow runs. The model download plus the subprocess round-trip means nothing happens for a while, then it grinds. That's expected, not a hang.
• The environment. If ComfyUI fails to start after install, it's almost always a dependency collision from this pack's very grabby requirements list - sort that before touching the node itself.
• Not the same as AIFSH/ComfyUI-Hallo - a separate wrapper by a different author that also shows up in Manager. Different repo, same underlying model.

There's no maintaining fork or big community behind this one; it's a "works if it works" wrapper. Treat it as a way to evaluate Hallo inside ComfyUI, not as production tooling.