AIIA Ditto Sampler

Ditto is Ant Group's talking-head model - the kind that takes one photo of a person and animates their face to an audio track. Official Ditto runs through TensorRT, which is a whole afternoon of build pain. AIIA Ditto Sampler is the reason you might skip that: this pack reimplemented the pipeline in native PyTorch, so you get Ditto-quality lip sync by just wiring up two nodes.

How it works

You connect three things: a pipe from the pack's AIIA Ditto Loader (which downloads the ~1.2GB weights into ComfyUI/models/ditto/ automatically), a square ref_image with the face centered, and the audio that drives the mouth. Outputs are the animated images and the passed-through audio. Ditto was trained at 25fps, so keep fps at 25 - the README says the internal logic tends to lock to 25 regardless.

The sampling run itself is a diffusion pass (sampling_steps, default 50). What makes this node worth its long parameter list is the long-form reliability work:

• save_to_disk - Memory (Default) keeps frames in RAM (fine under ~1000 frames); Disk (OOM-Safe) writes frames as it goes and returns frames_dir, which plugs into AIIA Video Combine's frames_directory. That's the whole OOM story of this pack in one dropdown.
• seed - more than a randomness knob. On long audio, the node resets the seed after silence so every sentence generates under identical conditions. That's the fix for the classic "mouth drifts off-sync by minute three" failure.
• relax_on_silence - during pauses the face settles instead of frozen-chewing, and the model state gets reset at long silences to kill cumulative lip drift.

The parameter that actually matters most

crop_scale (default 2.3) is the "how much of the world around the face does the model see" control. It doesn't change output resolution - it changes how much context the face has. Higher (2.5) = more headroom for big head movement but softer features; lower (2.0) = crisp, tight face but you can crop a forehead off when the head moves. Standard talking-head video: leave it at 2.3. Passport-style headshot: 2.0.

The rest are one-line quality controls: emo for a base expression (Angry, Happy, Sad…), drive_eye toggles eye animation, blink_mode picks the blink cadence (Natural/Slow/Fast/None), blink_amp/mouth_amp scale how hard the eyes and mouth open, mouth_smoothing adds inertia so plosives don't flap the jaw, and hd_rot_p/y/r nudge head pitch/yaw/roll. speech_pitch is a fun one - positive values make the head look down while talking, which reads as a more natural broadcast delivery.

Install & troubleshooting

Install havvk/ComfyUI_AIIA via Manager or git clone into custom_nodes/, restart. First run pulls the Ditto weights; if auto-download fails, grab digital-avatar/ditto-talkinghead from HuggingFace into ComfyUI/models/ditto/ - keep the ditto_pytorch/ and ditto_cfg/ subfolders intact. A messy reference image (tiny face, off-center) is the #1 cause of bad output - crop the face centered with AIIA Image Smart Crop first. And if a long clip OOMs, flip save_to_disk to Disk mode and feed frames_dir into Video Combine instead of keeping everything in RAM.