ITL Breeze TTS Voice Clone

Zero-shot voice cloning is the reason people put TTS inside ComfyUI at all. You want your character to say the line, in the voice you cast, and then you want that audio to drive a talking head or land on a video timeline - without exporting to a web UI and back. This node is the straightforward version of that: give it a few seconds of reference audio, tell it exactly what that audio says, type the line you want spoken.

Three inputs do the real work.

The inputs that matter

reference_audio - a few seconds of the voice you're imitating. Mono is fine; stereo gets downmixed. The node writes it to a temp WAV (mono, float32) for the runtime to read, and cleans the file up afterwards, so you don't have to pre-convert anything.

reference_text - the exact transcript of that clip. This is the input people get wrong, and it's the difference between a clone and a mush. The model is conditioned on the pair (audio, its words), so if the text doesn't match what's actually said, you've handed it contradictory information and the voice degrades. The tooltip is admirably blunt about it: "Wrong text = wrong voice." Use ITL Whisper Transcribe on the clip and paste the result; don't hand-type it from memory.

text - what you want spoken. This is a multilingual model (English and Chinese) and it detects the language from the text itself, so you don't pick one. You can put performance cues inline: (laugh), (sigh), (clears throat) in English, [笑] and [叹气] in Chinese. They work, and they're a genuinely nice touch for dialogue instead of flat narration.

Then seed (default 42) and unload_after (default off). One output: audio, a standard ComfyUI AUDIO you can wire into Save Audio, a lip-sync model, or anything else that eats audio.

How it works, roughly

The node builds a request for exactly one mode - this is worth knowing because the mode is explicit rather than inferred from what you wired in, so the clone template can't be swapped for a design template mid-flight. It picks the template, prepares the inputs with your guidance scale, seeds all the RNGs (torch, NumPy and Python) with the last thing before generation, and streams out audio chunks that get concatenated into one waveform. If text or reference_text is empty, you get a named error rather than silence.

The interesting omission: there is no cfg_scale on this node. The clone template has no negative branch, so classifier-free guidance has nothing to guide toward, and the engine rejects any value other than 1.0. Design and Direction get the knob; Clone doesn't. That's correct behaviour, not a missing feature.

Where the seed bites

The voice is (reference audio + reference text + seed). Same three, same output - so the seed is how you A/B a line: hold it fixed, change one thing, compare. There's a ComfyUI trap on top of that, though: the control_after_generate widget next to seed fires after the run, so with randomization on, the seed showing in the box is the one that will be used next, not the one that just produced the take you liked. If you're trying to pin down a delivery, set the control to fixed before you start, not after.

Install

Manager → search ComfyUI-IntoTheLatent-Utils → install → restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/Into-The-Latent/ComfyUI-IntoTheLatent-Utils
pip install -r ComfyUI-IntoTheLatent-Utils/requirements.txt

The pack vendors its own copy of the Breeze model code, so nothing is pulled from GitHub at import time, and it deliberately doesn't declare a torch dependency - your CUDA torch build stays exactly as it is. First run downloads ~7.2 GB of weights into models/breeze_tts/Breeze-TTS-2/. This node does need an NVIDIA GPU: no CPU path, ~7.7 GiB VRAM resident.

Gotchas

Empty reference_text is the number one failure. It errors with a message telling you so. That message is a gift - go transcribe the clip.

Don't feed it a whole podcast as reference. A few seconds of clean, single-speaker audio is the shape it wants; a long clip with background music teaches it the music.

The model staying resident is the other one. If you're cloning a line and then rendering video in the same graph, set unload_after on, or wire the audio through ITL Breeze TTS Unload. Otherwise the ~7.7 GiB is still there when your video model asks for memory, and ComfyUI's model manager can't reclaim it for you.

Licensing, briefly: the Breeze weights are research and non-commercial. Voice cloning has a second, non-licensing ethics question attached - don't clone someone who hasn't agreed to it.