Chatterbox Voice Conversion 🗣️

Text-to-speech invents a performance. Voice conversion steals one. You hand ChatterboxVC a recording of somebody talking and a target voice, and it re-voices the clip while keeping the words, the pacing, the breaths, the pauses and the emotion exactly where the human put them. The timing of the original survives; only the timbre changes.

That distinction is the reason to reach for it over re-generating with TTS. If a TTS read came out flat or monotone - the standing complaint about cloned voices across every engine - you can't prompt your way out of it. But you can read the line yourself, with the timing and emphasis you actually wanted, and convert it into the character voice. Same trick for dubbing: perform the line in your own language and delivery, land on someone else's voice at the end.

How it works

This is the S3Gen side of Chatterbox doing flow matching: an acoustic model converts the source speech into a mel representation and a flow-matching decoder re-synthesises it conditioned on the target speaker, which is why the node gives you diffusion-flavoured knobs (n_timesteps, temperature on the initial noise, flow_cfg_scale for how hard it leans on the target voice) rather than TTS text parameters. Content comes from your recording; identity comes from the target.

Two things make it practical on real recordings:

• Chunking. Long audio is split before conversion and joined after, which bounds the peak memory of a single conversion. chunk_method decides where the cut lands: smart looks for silence near the ideal boundary so the join is inaudible, fixed cuts at exact intervals - cheaper, and you can hear it.
• Refinement passes. Each pass feeds the previous output back in as the next input. One pass is the default; the tooltip recommends a maximum of five, beyond which you get audible distortion. Because each iteration is deterministic, raising the count on a clip you already converted returns the cached result instead of redoing the work.

target_voice_audio is optional - leave it unconnected and you get the default voice from the model pack. So the node is useful even with no reference clip at all.

Inputs and outputs

Required: model_pack_name, source_audio (the speech to be converted), n_timesteps (diffusion steps for the flow-matching process, 10 by default - higher may improve quality, slower), temperature (randomness of the initial noise, 1.0 is standard), flow_cfg_scale (0.7 default; higher hews closer to the target voice at some cost to naturalness), use_watermark, refinement_passes, max_chunk_duration (30 seconds by default; 0 disables chunking and converts in one pass), and chunk_method. Plus the runtime trio: device, dtype, attention_mode.

Output is a single converted_audio socket - straight into a save-audio node, a video mux, or an audio-driven avatar.

If you're converting a long file and the run dies on memory, max_chunk_duration is the dial: peak memory is one conversion, and 5 seconds is the smallest useful value. Given the choice, leave chunk_method on smart - a boundary placed in a pause is free, a boundary placed mid-word is not.

Install

Manager → Install Custom Nodes → search ComfyUI-Chatterbox → Install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/wildminder/ComfyUI-Chatterbox.git
pip install -r ComfyUI-Chatterbox/requirements.txt

ChatterboxVC keeps the node id it has had since 1.x, and the pack's only added dependency is s3tokenizer - the vendored core does its audio DSP in pure numpy/torch and dropped librosa, scipy, omegaconf and conformer, and hands reference clips to the model in memory rather than through temp .wav files. You need ComfyUI >=0.37.0 (V3 node API) and Python >=3.10. Model packs land in ComfyUI/models/tts/chatterbox/ on first run, ~2.5 GB per pack, shared weights stored once and reused.

One upgrade note that matters specifically for this node: 2.0.0 fixed a leak where the voice-conversion model was never actually released after a run - the reference was cleared on the patcher instead of the wrapper, so every VC run leaked a copy. If you do VC in batches on a machine you leave running, that fix alone is worth the upgrade.

Where people get burned

Too many refinement passes. Five is the recommended ceiling. Past it you're not refining, you're accumulating artifacts. And if you bump the count on the same clip expecting a different result, remember the passes are deterministic and cached - you'll get the cached output back.

Wondering why it doesn't sound like the target at all. flow_cfg_scale up means more adherence to the target voice and less naturalness; if the voice is coming out generic, that's the knob, and a proper reference clip in target_voice_audio matters more than the number.

Clean source audio. The content and the delivery both come from your recording, so a noisy source gets converted faithfully, noise and all.

Old interpreters. s3tokenizer is the one dependency that has caused install failures - reported on Python 3.10 setups such as Stability Matrix, fine on 3.12. Worth knowing before you blame the pack.

Jumping from 1.x. The models directory moved from models/chatterbox_tts/ to models/tts/chatterbox/ in 2.0.0 with no shim. Move the folders across or delete the old directory yourself. Saved workflows still load - the node id and widget order didn't change.