MultiTalk Prompt Schedule

InfiniteTalk - the multi-speaker continuous generation mode in Kijai's ComfyUI-WanVideoWrapper - is a one-prompt-per-video kind of beast. It follows the reference image and the audio far more strongly than it follows your words, and the native | separator only maps one prompt per sliding window: you control windows, not timing. Want scene one to have Person 1 talking at the camera and scene two to have Person 2 answer while Person 1 nods? MultiTalk Prompt Schedule lets you assign prompts to exact frame ranges, synced to the audio position instead of the window counter.

How it actually works

This is the part that makes people nervous, and it's worth being straight about: the node monkey-patches WanVideoWrapper at startup. On import it rewrites the prompt-selection block inside multitalk_loop.py in memory, replacing the iteration-based picker with a frame-schedule-aware one. It never touches files on disk - patch.py edits the live module via compile/exec, not the source file, so the README's "zero-patch install" claim checks out. If it can't find the block because WanVideoWrapper updated, it logs a warning and falls back to original behavior. Your other workflows keep working either way.

At generation time the node takes your whole schedule, encodes every prompt plus every unique negative with T5 in one pass, and packs a frame schedule into text_embeds. During sampling, each sliding window checks audio_start_idx against that schedule and picks the right prompt. Because selection keys off audio position rather than window number, prompts stay roughly in sync even when window sizes and your frame ranges don't divide evenly.

The schedule is dead simple. One scene per line, and the numbers are durations in frames that accumulate: at 24fps, 120 is 5 seconds.

# intro
120: Person 1 speaks to the camera, person 2 listens
120: Person 2 responds with a smile, person 1 nods
120: Both look at the camera and smile

Lines starting with # are comments and get ignored. Per-scene negatives use |||, covered below.

The inputs and outputs that matter

Only a few of the five inputs are ones you'll actually touch. schedule is the multiline scene list - the thing you'll be editing constantly. t5 takes the text encoder from the WanVideo T5 Text Encoder Loader. negative_prompt is the default NAG negative for any scene without its own |||. force_offload (default true) and device (gpu/cpu) are for managing T5 VRAM; leave them alone unless you're running tight on memory and the encoder is stomping on your generation VRAM.

The two outputs both feed the same WanVideoApplyNAG node: text_embeds → original_text_embeds, and negative_text_embeds → nag_text_embeds. That's the whole wiring - no separate negative text encoder node needed.

Why NAG? Standard InfiniteTalk runs at cfg=1, and at CFG 1 regular negative_prompt_embeds are inert - the only negative guidance that does anything is NAG (Normalized Attention Guidance), which operates inside cross-attention. So per-scene negatives are your lever for gaze control: a scene where characters look at each other gets ||| looking at camera, staring at viewer, and one where they face the camera gets ||| looking sideways, turned away. Scenes without ||| inherit the global negative_prompt. Keep the community's hard-won rule in mind: phrase NAG negatives as what the character doesn't do ("looking sideways") rather than forbidden actions - Wan tends to do the thing you forbid.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/serious-factory/ComfyUI-MultiTalkPromptSchedule

Then restart ComfyUI. It's also searchable in ComfyUI Manager as "ComfyUI-MultiTalkPromptSchedule" (note the hyphen; the Python module tries both hyphen and underscore import paths, which tells you the author has had to deal with both). It has no Python deps of its own - pyproject.toml declares none - but it is hard dependent on ComfyUI-WanVideoWrapper being installed first. Without it the node loads and the patch silently skips, and you'll wonder why nothing is scheduled.

Where people get burned

• Temper your expectations about prompt impact. InfiniteTalk follows reference image + audio more than text; gaze tweaks may barely show. The README's advice: crank nag_scale to 15–20 for more adherence, at some cost to visual quality.
• The re-run frozen-video trap. The patch also deep-copies audio_embedding before the loop - the original code aliases it, and on re-runs the cached (already-padded) embedding corrupts the size and you get a frozen frame with audio. If you've ever had an InfiniteTalk re-run come back frozen, this is that bug, fixed in-memory.
• WanVideoWrapper updates. A new version that reshuffles the prompt-selection source makes the patch miss. It degrades gracefully, but scheduling stops. Reinstall this pack or check its log when that happens.

If you just want one prompt per window, you don't need this node. If you want actual scene timing - who's talking, who's listening, whose eyes are where, synced to the audio - this is the cleanest way to get it without hand-editing the wrapper.