⚙️ CosyVoice3 Engine
The fast multilingual cloner with instruct mode
- TTS_engine
CosyVoice3 is Alibaba's speed play. It's a small (~0.5B, ~5.4GB) model that generates absurdly fast - roughly 20x faster than real-time, so ten seconds of audio comes out in about half a second. If you're iterating on a line over and over, that turnaround changes how you work. It clones a voice zero-shot from a 3–30s reference, covers English/Chinese/Japanese/Korean, and has a couple of tricks bigger models don't: native paralinguistic tags (<laughter>, <breath>, <sigh>) and an "instruct" mode where you steer delivery with a plain sentence.
Config node, as always - it outputs a TTS_engine you plug into 🎤 TTS Text or 📺 TTS SRT.
How it works
You choose a checkpoint, optionally write an instruction, and let it clone from whatever reference voice the Text/SRT node provides. The instruct mode is the fun part: instead of hunting for the right tag, you can literally write "Speak with a joyful tone" or ask for a Cantonese dialect, and CosyVoice adjusts. Paralinguistic tags in the text (like <laughter> or <laughing>text</laughing>) are processed during generation, so laughs and breaths land in the flow rather than being pasted on afterward.
The inputs and outputs that matter
model_path-Fun-CosyVoice3-0.5B-RL(the default, RL-tuned) or the plainFun-CosyVoice3-0.5B. Start with RL.speed- 0.5x to 2.0x delivery speed. A rare engine where speech rate is a first-class dial.instruct_text(optional) - your natural-language delivery instruction. Leave blank for straight cloning; fill it to direct tone/dialect/emotion.use_fp16(default on) - half precision, keeps it fast and lean. Fine to leave.
The load_trt and load_vllm options are optional acceleration backends (TensorRT / vLLM). They can speed things up further but add setup friction and aren't needed to get going - leave them off unless you've deliberately set those up.
Output: TTS_engine → TTS Text or TTS SRT.
Installing it
Bundled. ComfyUI Manager → "TTS Audio Suite" → install → restart. Manual:
cd ComfyUI/custom_nodes
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
cd TTS-Audio-Suite
python install.py
CosyVoice weights (~5.4GB, variant-specific lazy downloads) auto-download to ComfyUI/models/TTS/CosyVoice/ on first use.
Common issues & troubleshooting
Your instruction was ignored. Instruct mode is a CosyVoice feature, but make sure you're actually putting the direction in instruct_text and not in the spoken text - the spoken text is what gets read, the instruction is what shapes how. Keep them separate.
Tags read out literally. Use the exact tag forms the model supports (<breath>, <laughter>, <sigh>, <laughing>...</laughing>). A typo'd or unsupported tag can leak into the audio as spoken words.
Wrong language. CosyVoice3 covers four core languages well - English, Chinese, Japanese, Korean. Push it outside those and quality drops; use language tags ([en:], [zh:], [ja:], [ko:]) or native <|en|> markers to be explicit rather than relying on guesswork.
load_trt / load_vllm errors. Those backends need their own dependencies and compatible hardware. If they fail to load, turn them off - the default fp16 path is already fast, and it's what most people should run.
Inputs (7)
| Name | Type | Default | Description |
|---|---|---|---|
| model_path | COMBO | Fun-CosyVoice3-0.5B-RL | CosyVoice3 model variant: • Fun-CosyVoice3-0.5B-RL: Reinforcement learning trained variant with improved quality (0.81 CER, 77.4 speaker similarity - recommended) • Fun-CosyVoice3-0.5B: Base model • local:ModelName: Use locally installed model |
| device | COMBO | auto | Device to run CosyVoice3 model on: • auto: Best available (CUDA > CPU) • cuda: NVIDIA GPU • cpu: CPU-only processing (slower) |
| speed | FLOAT | 1.00.5–2 | Speech speed multiplier (0.5=slow, 1.0=normal, 2.0=fast). IMPORTANT: CosyVoice's official implementation uses mel-spectrogram interpolation for speed control - this is post-processing time-stretching applied AFTER generation, NOT natural prosody/rhythm changes during synthesis. This is a limitation of the official model architecture. Result will sound like artificially sped up or slowed down audio (like playing a recording faster/slower). |
| use_fp16 | BOOLEAN | true | Use FP16 for faster inference. Disable if you encounter numerical issues. |
| instruct_textopt | STRING | Optional instruction for dialect/emotion/speed control. ⚠️ IMPORTANT: Instruction mode is mutually exclusive with transcript-based voice cloning. • When instruction is provided: Uses instruction mode (ignores transcripts) • When empty: Uses zero-shot cloning with transcripts (.txt files) for BEST QUALITY Examples: • 请用广东话表达。 (Use Cantonese dialect) • 请用四川话说。 (Use Sichuan dialect) • 请用尽可能快地语速说一句话。 (Speak as fast as possible) • 请用温柔的语气说。 (Use gentle tone) • 请用生气的语气说。 (Use angry tone) For best voice cloning quality: Leave this empty and provide .txt transcripts via Character Voices | |
| load_trtopt | BOOLEAN | false | TensorRT for flow decoder (FP32 only, <1% speedup). Included for compatibility with official repo but provides negligible benefit. Use load_vllm for real LLM acceleration. |
| load_vllmopt | BOOLEAN | false | vLLM for LLM acceleration (~3x faster token generation, ~2.5x total speedup). Linux only - does not work on Windows. Requires vLLM installation and model conversion. |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| TTS_engine | TTS_ENGINE | — |