SONICSampler

Don't let the name fool you. SONICSampler is not one of ComfyUI's normal KSamplers - there's no prompt, no positive/negative conditioning, no CFG on the widget. It's the payoff node in a three-node chain (SONICTLoader → SONIC_PreData → SONICSampler) that animates a still portrait so it talks, sings, and moves in sync with an audio track. Sonic is the research method behind it - "Shifting Focus to Global Audio Perception in Portrait Animation" (arXiv 2411.16331) - and the point of that name is that it conditions on the whole audio signal, not just the lips, so you get head motion, expression, and eye movement matching the speech instead of a frozen face with a wobbling mouth.

This node is where the video actually gets made. Everything before it just gets the ingredients ready; this is the denoising run that takes minutes-to-an-hour of GPU time and spits out the frames.

What it takes in

Only two data inputs, which is nice:

• model (MODEL_SONIC) - straight out of SONICTLoader.
• data_dict (SONIC_PREDATA) - straight out of SONIC_PreData. This carries the audio embeddings, the motion buckets, and the pre-encoded image latent, all precomputed so the sampler can focus on denoising.

Then the knobs:

• seed - the usual noise seed. Same seed + same inputs = same video.
• inference_steps (25 default) - more steps, crisper result, more time. 25 matches the pack's own config, so don't feel compelled to crank it.
• dynamic_scale (1.0, range 0.5–2.0) - this is the motion bucket scale, not CFG. It multiplies how much motion the model targets for each window. Above 1.0 gets you more energetic, expressive movement; below 1.0 calms it down. Good place to start if output looks too twitchy or too stiff.
• fps (25, range 5–120) - the output frame rate.

What comes out

Two outputs: image - one IMAGE tensor holding all the frames in batch, which is why the example workflow feeds it straight into CreateVideo → SaveVideo - and fps - a pass-through of the fps you set, so the video writer knows what to call it.

The gotchas that actually bite

• This is the slow, expensive step. On a 3090, one r/comfyui user reported about 80 minutes for a short clip at 448px resolution. Budget accordingly, and don't be surprised when a "30-second video" takes the rest of your evening.
• The default workflow does not save the audio with the video. The image output is frames only; you re-mux the audio in your editor (or wire it in however your SaveVideo setup allows). Easy to forget the first time.
• OOM lives here, but you fix it upstream. The README is blunt: lower min_resolution in SONIC_PreData, not the steps here. First run on 12GB VRAM can also OOM while models warm up - the author patched that, but big resolutions will still find your limit.
• Keep fps consistent. There was a batch-mismatch bug when the frame rate wasn't 25; it's fixed, but don't set 25 here and 30 in CreateVideo and expect clean results.

Install

This pack is one of smthemex's many ComfyUI ports. Install it once for all three nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_Sonic.git
pip install -r requirements.txt

Or search "ComfyUI_Sonic" in ComfyUI Manager and let it handle it. The dependencies are the heavy part (diffusers, transformers, torchaudio, librosa, opencv), and there's a known crash on torch 2.11 + cu130 where torchaudio.save blows up via a torchcodec ABI mismatch - a maintained fork (svilendotorg/ComfyUI-Sonic) swaps that call for soundfile.write. If you hit that, use the fork. Then download the model files per the README's Google Drive link into ComfyUI/models/sonic/.

Once it's running, expect some trial-and-error on dynamic_scale per input - that's normal. The good news from the community: when people get Sonic working, they describe it as "a chore it wasn't."