IMTalker Audio Driven

Feed it one photo and one audio file, and you get a video of that face lip-syncing to the audio. No API key, no cloud round-trip - this is a local talking-head model from the CBS lab at Shanghai Jiao Tong University (IMTalker, an ACM MM 2025 paper on "efficient audio-driven talking face generation with implicit motion transfer"), wrapped into a ComfyUI node by tpc2233. If you've seen the modern avatar stack - Chatterbox or Kokoro for the voice, then an audio-driven model to make the face move - this is the second half of that pipeline.

Be clear about what it is not. This isn't Wan or LTX doing generative video; it doesn't invent audio and it won't render new scenery. It consumes your audio as a driving signal and transfers motion onto a fixed identity. Output is 256×256, 25fps frames of that one face. For a talking-head or lip-sync shot that's exactly what you want - cheaper and far more controllable than a video base model.

How it works

The pipeline: audio → wav2vec2 features → flow-matching generator → renderer → frames.

Your audio gets mixed to mono, resampled to 16kHz, and tokenized through wav2vec2's feature extractor. That's not just for show - the generator's own audio encoder is a frozen Wav2VecModel subclass reading those same downloaded weights. Those features drive a flow-matching transformer, the FMGenerator, which samples a motion latent by integrating an ODE with torchdiffeq (Euler steps). The renderer then decodes the latent into frames. This is real flow matching, so the steps you set are ODE function evaluations, not diffusion denoising steps - 10 is a sensible default, and cranking to 100 mostly buys you wall time, not quality.

Clip length tracks your audio: a five-second WAV gives you about five seconds of video (T = audio samples × 25 / 16000).

The inputs that matter

Only a few are worth touching:

• image - your talking head's face. A clean, front-ish portrait works best.
• audio - the ComfyUI AUDIO type, straight from LoadAudio.
• cfg_scale (default 3.0) - audio classifier-free guidance. Higher pushes the motion to track the audio harder; lower is calmer. The one knob you'll actually play with.
• steps (default 10) - flow-matching function evaluations. Leave it until you're debugging.
• seed (default 42) - noise seed for the flow sampling. Same seed, same motion.
• crop_face (default true) - face-alignment's FAN detector crops a square around the face before the 256px resize. No face found? It falls back to a center crop and warns.

It also takes the model_bundle from IMTalkerLoader - forgetting that wire is the easiest way to get a red node.

The output

One IMAGE output - a batch of silent 256×256 frames. Wire them into VHS_VideoCombine (frame rate 25) and feed the original audio into that node's audio input to end up with a talking video with sound. That's exactly what the pack's bundled workflow does. No VideoHelperSuite installed? SaveAnimatedWEBP works, but you lose the audio.

Install and gotchas

ComfyUI Manager: search "ComfyUI-TP-IMtalker". Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/tpc2233/ComfyUI-TP-IMtalker.git
cd ComfyUI-TP-IMtalker
pip install -r requirements.txt

The loader auto-downloads the model files to ComfyUI/models/imtalker on first run - a few gigabytes, so the first queue sits on a progress bar. Real talk on hardware: the README quotes up to ~17GB VRAM for this (tested on an RTX 6000 Pro, torch nightly cu128). If you're on 8GB, check your card before committing.

Two things that bite people:

• torchaudio is imported but not in requirements.txt. nodes.py imports it at the top, so if your ComfyUI env lacks it, the whole pack fails to register and there are no IMTalker nodes in your menu at all. The author's tested install installs it explicitly: pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128. Empty menu? That's the first suspect.
• Transformers version churn. The loader monkey-patches Wav2Vec2Model.from_pretrained to force attn_implementation="eager", dodging a classic attention error on newer transformers. If you hit a transformers error in the generator, pin or update transformers - this seam is the whole hobby in miniature.

Face-alignment also downloads its own weights on first use and drags in a heavy dependency tree, so first installs take a while.

Bottom line: load the models, wire an image and an audio, save the video. It's a niche tool - a 256px single-face animator is not the general video model everyone's chasing - but for dubbing, avatars, or animating one portrait, it does one job well, entirely on your own GPU.