XB-BOX - 💬 CosyVoice3 多人对话

The reason to run a local voice-clone suite instead of an API is usually a conversation scene: you need two distinct voices trading lines, and you want it in one take, in one audio file, without stitching clips by hand. XB_CosyVoice3_Dialog is the multi-speaker node in XB_ToolBox's CosyVoice3 suite that does exactly that - you write the script with speaker labels, give each speaker a short voice reference, and it synthesizes the whole dialogue and hands you a mixed track plus each speaker's isolated audio.

This is the node the pack's audio-slicer family is designed to feed into: slice your references with XB_AudioSlicer, feed the cleaned samples here, and out comes the conversation.

How it works

You write the script in dialog_text with SPEAKER A:, SPEAKER B: prefixes (the label format is literal - the code splits on them):

SPEAKER A: Hello, how are you?
SPEAKER B: I'm doing great, thanks for asking!

Each line is synthesized in the matching speaker's cloned voice - speaker_A_Audio / speaker_B_Audio are required references, and speaker_C_Audio / speaker_D_Audio let you extend it to four voices. The node validates each reference's duration (it raises a clear error if a reference is too short - the family's floor is roughly half a second, with 3–10s recommended), synthesizes line by line, and mixes the result into a single dialog_audio track. speed and seed apply across the whole dialogue.

What makes it genuinely useful is the outputs: besides the mixed dialog_audio, you get speaker_a_audio / speaker_b_audio / speaker_c_audio / speaker_d_audio - each speaker's lines as a separate track. So you can take the mix for a quick listen and the isolated stems for proper audio post - EQ, leveling, or cutting one speaker entirely.

The inputs that matter

• model - a COSYVOICE_MODEL from XB_CosyVoice3_ModelLoader. Required.
• dialog_text - the script, SPEAKER X: prefixes per line. Label order doesn't matter; the node maps lines to whichever speaker audio you provided.
• speaker_A_Audio / speaker_B_Audio - the two reference voices (required).
• speaker_C_Audio / speaker_D_Audio - optional third/fourth voices.
• speed - 0.5–2.0, default 1.0.
• seed - 42 default, -1 for random.

Outputs: dialog_audio (mixed), speaker_a…d_audio (stems), message (a STRING status/log you can wire to a text preview to see what the node did).

Installing

Same pack path:

cd ComfyUI/custom_nodes
git clone https://github.com/WJLUOXIAO/XB_ToolBox.git

restart, then install the CosyVoice3 dependency stack from requirements.txt (transformers, librosa, soundfile, modelscope, conformer, x-transformers, diffusers, pyworld, …). The ModelLoader pulls the model on first use - Fun-CosyVoice3-0.5B by default, from HuggingFace or ModelScope. Dependency conflicts with ComfyUI's own transformers are the usual failure point; a clean env for the pack fixes most of it.

Common issues

• "Reference audio too short" error - the node validates each speaker's clip (it needs ≥ ~0.5s). Crop longer, cleaner samples upstream with XB_CosyVoice3_AudioCrop.
• All lines in the same voice - you probably fed the same reference to both speakers, or wired one speaker_*_Audio into the wrong slot. Each speaker needs its own distinct reference.
• A line assigned to the wrong speaker - check your SPEAKER X: labels are exact (the code matches SPEAKER A:, SPEAKER B: prefixes).
• Mixed track clips/distorts - the mix is a simple sum; if two speakers overlap loudly, reduce overall levels in post (you have the stems for exactly this).

Honest note: voices will sound like the same model speaking two people unless your references are clearly distinct - CosyVoice clones the timbre well, but two similar-sounding references will produce two similar-sounding voices. Choose your reference clips to be as different as the characters are.