XB-BOX - 🎭 CosyVoice3 说话人指令

SpeakerClone gives you a voice. This node gives the voice a personality. SpeakerInstruct2 is the same saved-preset synthesis, but with an extra field - instruct_text - where you tell the model how to deliver the line. "Say this sadly." "Say this very excitedly." "Whisper it." That one field is the difference between a robotic narration and a line that actually lands.

This is the combination people end up living in for character work. You save one voice for the character, then use this node for every line, varying the instruction to get anger, tenderness, exhaustion, whatever the scene needs - all in the same timbre. For a digital-human or lip-sync pipeline it's borderline essential, because an emotionless take reads as uncanny the moment a face is attached to it.

How it works

Under the hood it's the SpeakerClone machinery with one upgrade: it calls the model's inference_instruct2 with a zero_shot_spk_id instead of plain inference_zero_shot. That's CosyVoice's instruction-guided synthesis path - the text controls what is said, the instruct text controls how. The speaker embedding still comes from your saved .pt preset, so the timbre stays locked to the voice you captured, and no reference audio is needed in the workflow.

The instruct text is prompt-engineering-adjacent, and CosyVoice is a Chinese-first model, so the default is a Chinese example - 请非常开心地说这句话。 ("please say this very happily"). English instructions work too; the model's instruction-following is decent in both.

The inputs that matter

• instruct_text - the delivery instruction. This is the star of the show. Being specific helps: "say this in a very soft, sad voice" beats "sad".
• speaker_preset - the saved voice from SaveSpeaker. Same [none] placeholder trap as SpeakerClone if you haven't saved one yet.
• text - the actual line to speak.
• speed - 0.5x to 2.0x.
• seed - default 42; keep it fixed while you tune the instruct text so you're comparing like for like.
• text_frontend - on by default, off for phonemes/special tags.

Output: one audio (AUDIO) wire, into a preview, saver, or downstream video pipeline.

Install and gotchas

Pack install is the standard one - ComfyUI Manager search XB_ToolBox, or git clone into custom_nodes and restart, with the CosyVoice requirements.txt deps installed. Freshly saved speakers need a workflow reload to appear in the dropdown, same as SpeakerClone.

The real tuning notes: instruct text is not a magic knob. Very short, concrete instructions ("whisper", "excitedly, like a child") tend to work better than paragraphs, and the emotion control is strongest when the instruction and the text are in the same language. Chinese instructions on English text can give you a confused accent - worth knowing since the default is Chinese. And if an instruction does nothing, it's usually because the model needs a stronger contrast: "very soft, almost a whisper" will move the needle more than "soft".

One more practical tip: when you find an instruct text that works for a scene, save it. There's no preset store for instructions, so a snippet file of your best ones saves a lot of retyping.