WAN LipSync Timeline Planner

WAN's InfiniteTalk workflow generates a talking-head video from a single reference image and an audio track. The Eclipse twist is letting you do it with several reference images - shot changes, expression changes, a whole mini-scene - and this node is the thing that figures out when each image takes over. It turns your audio plus an ordered stack of reference images into a frame-exact plan of "generate this segment with image #1, switch to image #2 here, hold it there."

What it builds

Think of it as a planner, not a generator. It produces a plan object - the WAN_LIPSYNC_PLAN - that describes the whole timeline as a list of contiguous extension tasks, each with a start frame, an end frame, which reference image it uses, and whether it begins with a transition. It also hands you task_count, total_frames, base_keep_frames (the frames carried from the first image before any switch), and a report string you can dump to Show Text to see what it decided.

The inputs it plans from:

• audio - defines the final timeline length. The total frame count is derived from the audio duration and fps.
• image_batch - your ordered reference images. The node needs one "switch time" for each image after the first.
• context_length (default 81) and overlap_frames (default 9) - the fixed-length generation window and the overlap the sampler carries between chunks. These should match how your WAN sampler actually runs; the plan splits the timeline into chunks that fit inside context_length - overlap.
• manual_transition_times - comma-separated switch times in seconds, one per image after the first. Leave blank and the node spaces transitions evenly across the timeline.

The silence-alignment feature

Here's the clever part and the reason this node is fiddly. If align_to_silence is on (default), the planner doesn't just honor your requested switch times - it searches near each target for a gap in the audio's activity, so the image change lands between words rather than mid-syllable. It reads the Wav2Vec2 encoder output (the same one InfiniteTalk already computes; pass it in via audio_encoder_output so it doesn't recompute) and treats energy gaps as activity boundaries. The knobs search_window_seconds, min_silence_duration, and resume_hold_duration control how far to look, how quiet a gap must be, and how long speech must resume before the transition counts.

If you'd rather have exact frames, flip align_to_silence off and the planner honors your manual times to the frame.

Where it sits in the workflow

Planner first, then WAN LipSync Plan Step in a loop. The plan feeds Plan Step, which pulls out one task at a time for the sampler to generate - so the pair is: plan the whole video once, then execute it chunk by chunk. For a beginner the takeaway is: get your audio, get your reference images in order, set context_length to match your sampler, and let the defaults do the rest. The silence alignment defaults (1.5s window, 0.5s min silence) are sane starting points.

Install it

Part of ComfyUI_Eclipse. ComfyUI Manager → search "Eclipse" → Install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/r-vage/ComfyUI_Eclipse

It's under Eclipse → Video. Standard pack deps (opencv-python, pilgram, PyYAML, aiohttp; portable installs may need pip install -r custom_nodes/ComfyUI_Eclipse/requirements.txt).

Gotchas

The plan is metadata - it does nothing until Plan Step executes it, so a lonely planner in a workflow is a silent no-op. And it's strict: if you give it two transition times but three images after the first, it errors rather than guessing. The most common beginner mistake is changing fps here without changing it in the sampler - the frame math in the plan assumes one consistent fps, and a mismatch makes the switches land off-beat. Finally, if your audio is too short for the number of images and overlap spacing, the planner refuses with "audio is too short" - that's not a bug, that's physics; cut images or shorten the overlap.