XB-BOX - 🎵 语音转视频分块

Every relay in the pack's InfiniteTalk pipeline eventually calls down to one node that actually does the audio-to-video work: XB_WanInfiniteTalkToVideo ("XB-BOX - 🎵 语音转视频分块"). This is the "voice to video chunk" engine - it takes an audio encoder output, a start image, and conditioning, then builds the model patch and latent that make the character's lips and motion follow the audio for one chunk of video.

The author's own description in the source is "兼容旧工作流" - a compatibility node for older workflows - because it's the do-everything version: a single mode toggle switches between single-speaker and two-speaker operation. The pack also ships the split single/dual nodes (..._Single and ..._Dual) if you prefer fixed, unambiguous sockets.

How it works

Everything funnels through one shared audio-processing helper (_process_infinite_talk_audio), which does the real work:

1. Clones the model - all audio patches are applied to a clone(), so a cancelled run or OOM leaves the original model unpolluted.
2. Pins the start image - VAE-encodes it into concat_latent_image + concat_mask so frame one is locked (standard Wan I2V bookending).
3. Builds the audio conditioning - stacks the audio_encoder_output's per-layer embeddings, downsamples from 50fps to 25fps feature rate, and writes it into the conditioning as the audio drive. For two speakers, both audio_encoder_output_1 and _2 are encoded and combined; the node refuses to run two-speaker without both masks.
4. Handles the overlap - if you feed previous_frames, they're checked against motion_frame_count (it raises if you give fewer frames than the overlap), giving the model the tail of the last segment so the transition is continuous.

Outputs: model (the patched clone), positive, negative, latent, and trim_image - an INT telling the relay how many overlap frames to trim off the start of this chunk.

Inputs that matter

mode (single_speaker default / two_speakers) is the fork in the road. audio_encoder_output_1 is the required audio; _2 plus mask_1/mask_2 are required for two speakers (masks say who's who in the frame). motion_frame_count (9) is the transition overlap, audio_scale (1.0) the audio-to-motion strength. vae_tile_size (256 default here) controls encode memory.

Install & honest caveats

Pack standard: ComfyUI Manager → XB_ToolBox, or git clone https://github.com/wjluoxiao/XB_ToolBox.git into custom_nodes, restart. Deps opencv-python + easyocr; weights are yours (InfiniteTalk checkpoint, Wan audio encoder, VAE/CLIP/T5).

This node does not sample - it returns the patched model, conditioning and latent, which you feed to a KSampler (or let a relay handle it). And it's a segment engine: 81 frames in, 3-ish seconds out. InfiniteTalk is a genuine open talking-head model with real community success stories, but if you're here, you've committed to the audio-driven niche - the rewards (characters who talk for as long as your audio) are worth the plumbing.