Dice_Talk_PreData

If Dice_Talk_Loader assembles the model, this node prepares everything the model needs to actually talk. The Loader gave you a weight_dtype; now you hand it your audio, your photo, a CLIP vision encoder, a VAE, and an emotion - and it returns one data_dict that the Sampler consumes whole. Nothing you wire into PreData is a "prompt." Everything you wire into it gets processed into the latent-space conditioning that drives the video.

What it's doing under the hood

Four jobs, run in sequence. First, it runs your audio through whisper-tiny's encoder to produce audio features (don't expect a transcript - it's using the encoder, not the decoder, so no text ever appears). Second, it runs face detection with yoloface and crops/aligns your photo so the video has a stable face to animate. Third, it loads your chosen emotion prior and the emo_model.pth + audio_linear.pth weights to build the audio and emotion embeddings. Fourth, it embeds the reference image through CLIP vision and VAE-encodes the source photo into latents. All of that gets bundled into the single DICETALK_PREDATA output.

It's also deliberately tidy: when it's done, it moves the heavy audio/emotion models to CPU and calls unload_all_models(), freeing VRAM for the render ahead.

The inputs that matter

• clip_vision, vae - from the same ImageOnlyCheckpointLoader that feeds the Loader. If you loaded the SVD checkpoint a different way, make sure CLIP vision and VAE are the SVD ones.
• audio, image - LoadAudio and LoadImage, the boring ComfyUI core nodes. The image wants a clear, front-facing face; this node will hard-fail otherwise.
• emo_files - the dropdown with 8 emotion priors shipped inside the pack: neutral, happy, angry, sad, surprised, disgusted, fear, contempt. This is DICE-Talk's party trick - you're not hoping for emotion, you're picking it. Neutral is the safe start.
• duration - how many seconds of video to synthesize, capped at the actual audio length. If your clip is 8 seconds and you set 10, you get 8.
• min_resolution - the working resolution of the face crop, 512 default. This is your main VRAM/speed lever: drop to 384 or 448 to fit smaller cards, like people end up doing with the whole Sonic/SVD family.
• expand_ratio - how much padding around the detected face to keep in the crop, 0.5 default. More headroom, less tight framing.
• retrieval - off by default. Turns on the emotion-retrieval mode from the paper, which pulls the emotion prior from a reference rather than the fixed file. Leave it off until the basics work.

What comes out

One output, data_dict of type DICETALK_PREDATA, straight into the Dice_Talk_Sampler's data_dict input. Don't preview it, don't wire it anywhere else - it's an opaque bundle of tensors.

Installing and model files

Same install as the rest of the pack (clone https://github.com/smthemex/ComfyUI_DICE_Talk, pip install -r requirements.txt, restart - see the Loader article for the full steps). This node specifically needs audio_linear.pth, emo_model.pth and yoloface_v5m.pt in ComfyUI/models/dice_talk plus the whisper-tiny folder, and it checks for them up front with its own "Please download the model first" error. If you run smthemex's ComfyUI_Sonic, its whisper-tiny and yoloface are reused automatically.

The gotchas

The big one is the "No face detected" ValueError: the face detector either didn't find a face or the crop math went wrong. Fix it at the source - brighter, more frontal, less occluded input photo. Second, this node is slow and momentarily memory-hungry: whisper encode plus face align plus VAE encode all happen here, so the first run after loading the model feels like a hang. It isn't; watch the console. Third, the whisper/yolo/emo models are loaded fresh each run, so the whole setup repeats every generation - don't fight it, just don't expect fast iteration between takes.