XB-BOX - 🎭 CosyVoice3 指令控制

Zero-shot cloning gives you the voice, but not the performance. "Speak in a warm and friendly tone," "slow down and whisper," "read it like an excited announcer" - that's instruction-based control, and it's CosyVoice's most underrated trick. XB_CosyVoice3_Instruct2 wraps it: you give it a cloned reference voice, the text to speak, and a plain-English instruction describing the delivery, and out comes the audio. No emotion tags to memorize, no per-parameter fiddling - you describe the performance and the model does its best.

It's the second-gen instruct node in XB_ToolBox's CosyVoice3 suite (hence the "2"), sitting alongside zero-shot cloning and cross-lingual synthesis. For voice acting in generated videos - a narrator, an angry line, a tired response - this is the node that turns "a voice" into "a take."

How it works

Like the rest of the family, it takes a COSYVOICE_MODEL from the pack's ModelLoader, clones the speaker from reference_audio, and synthesizes text. The difference is the instruct_text field: a natural-language description of tone, emotion, speed, and delivery that gets mixed into the synthesis conditioning. The node's own docs suggest examples: "Speak slowly and gently," "Use an excited and energetic tone," "Sound calm and professional."

Supporting knobs:

• speed - 0.5–2.0×, default 1.0. Also describable in instruct_text ("speak slowly"), but the explicit control is more predictable.
• seed - default 42; -1 random. Instruction models are seed-finitely too - same instruction, different seed, noticeably different take.
• text_frontend - text normalization on by default. Turn off for CMU phonemes or raw tags like <slow>.

Output: audio - the synthesized clip.

The inputs that matter

• model - from XB_CosyVoice3_ModelLoader. No standalone loading here.
• text - what to say.
• instruct_text - how to say it. This is the star of the show; be specific, one or two instructions read better than a paragraph.
• reference_audio - the voice to clone, clean, 3–10s recommended (validated, max ~30s).

Installing

In XB_ToolBox:

cd ComfyUI/custom_nodes
git clone https://github.com/WJLUOXIAO/XB_ToolBox.git

restart, then the CosyVoice3 dependency stack from requirements.txt (transformers, librosa, soundfile, modelscope, conformer, x-transformers, diffusers, pyworld, …). First model load downloads Fun-CosyVoice3-0.5B from HuggingFace/ModelScope - a few GB on the first run. The dependency dance (especially transformers pinning) is the #1 reported failure; a dedicated venv for the pack sidesteps it.

Common issues

• Instruction ignored / barely different - instructions steer delivery, they don't puppeteer it. Short, concrete instructions ("whisper", "excited") work better than abstract ones ("with deep emotion"). Also try another seed.
• Reference too long or too short - validated (≤ ~30s, needs ≥ ~0.5s). Crop to the sweet spot with XB_CosyVoice3_AudioCrop.
• Reads numbers wrong - keep text_frontend on unless you deliberately want phonemes.
• Sounds robotic at extreme speed - push speed past ~1.5 and artifacts creep in; prefer describing pace in instruct_text and keeping speed near 1.0.

Honest verdict: it's a directional control, not a guarantee - "warm and friendly" won't reliably transform a monotone reference into a Disney take. But for getting energy, pace, and tone roughly right in one shot, it beats hand-tuning or re-recording, and it's a genuinely fun node to play with once your CosyVoice stack is up.