SONIC_PreData

SONIC_PreData is the middle node of the ComfyUI_Sonic chain (SONICTLoader → SONIC_PreData → SONICSampler), and it's the one doing the unglamorous work that decides whether your talking-head video looks right. It takes the portrait, the audio, and the SVD model's CLIP vision + VAE, and chews them into the data_dict the sampler consumes. Nothing about it generates pixels - but everything about it determines whether the pixels are any good.

What it actually does

Under the hood this node is a small pipeline: it runs OpenAI's whisper-tiny over your audio to extract speech features, chunks them into ~3-second windows, feeds them through audio2token (an audio→prompt projector) and audio2bucket (which predicts a motion intensity value per window, scaled up to feed SVD's motion bucket), runs yoloface face detection to find, align, and crop the face, encodes the reference image with CLIP vision, and VAE-encodes it into the latent the sampler starts from. All of that gets packed into one SONIC_PREDATA object. That's why the node feels heavy on first run - it loads whisper, both audio adapters, and the face detector fresh every time, then moves them off the GPU when it's done.

The inputs that matter

There are eight, and honestly only two are for you:

• clip_vision, vae - both come from the same ImageOnlyCheckpointLoader you feed your SVD checkpoint into (or dedicated CLIP vision / VAE loaders).
• audio (AUDIO) - from a LoadAudio node. This is the speech your portrait will lip-sync to.
• image - a still portrait. It needs a detectable face; the yoloface model does the finding.
• weight_dtype (DTYPE) - the second output of SONICTLoader. Wire it through so everything runs at the same dtype.
• min_resolution (512, range 128–2048) - the minimum output size. This is your VRAM lever: the README says in plain terms, if you OOM, lower this.
• duration (10s) - how many seconds of the audio to animate. It clamps to the actual length of the input file, so you can't exceed it.
• expand_ratio (0.5, range 0.1–1) - how much of the area around the detected face gets cropped as the animation region. Tighter crop = more focused on the face; looser = more context but more room for drift.

Output

One output: data_dict (SONIC_PREDATA) → into SONICSampler's data_dict input. That's the entire handoff.

Where people get burned

• "Please download the model first." The node raises this if audio2token.pth, audio2bucket.pth, or yoloface_v5m.pt aren't in ComfyUI/models/sonic/. Grab the whole folder from the README's Google Drive link; also drop whisper-tiny (from Hugging Face) into models/sonic/whisper-tiny/.
• A cannot import name 'cached_download' from 'huggingface_hub' error at import time is an old-revision symptom: your huggingface_hub is too new for the bundled transformers/whisper code. Update the pack or pin huggingface_hub to a version that still ships that symbol.
• The torchaudio crash. On torch 2.11 + cu130, the temp-WAV write this node does can hard-crash the worker through a torchcodec ABI mismatch. The maintained fork (svilendotorg/ComfyUI-Sonic) replaces it with soundfile.write; if you're on that torch, use the fork.
• duration isn't frame-perfect. The README notes the comparison is done on the audio amplitude array, not a precise percentage - so your clip length can be off by a hair. Don't panic; trim in the editor.

Install the pack with git clone https://github.com/smthemex/ComfyUI_Sonic.git into custom_nodes, pip install -r requirements.txt, and restart - or use ComfyUI Manager, searching "ComfyUI_Sonic". The models are the actual install. Once they're in, the node is genuinely set-and-forget: you tune min_resolution for VRAM, expand_ratio for framing, and let it work.