FishSpeech Voice Clone

FishSpeech Voice Clone is the reason to install this pack. Feed it a few seconds of someone talking, the subtitle file for that clip, and an SRT of whatever you actually want them to say - and you get back audio that keeps the reference's rhythm while saying your words. That's the part plain TTS can't do. Most voice-clone nodes only match a timbre; this one matches the pacing too, so the result drops into a video timeline without you re-cutting everything. It's essentially a local, open-weight dubbing rig.

How it actually works

Under the hood this is fish-speech's v1.4 pipeline, vendored in and shelled out to as three steps per segment. First a VQGAN codec (vq-gan-group-fsq-2x1024.pth) turns your reference clip into semantic tokens. Then a text2semantic transformer - a dual autoregressive LLaMA-style model, in medium or large - generates new codebook tokens conditioned on your text plus the reference's tokens. Finally the same VQGAN decodes those tokens back into a waveform.

The SRT part is where the trick lives. Every subtitle cue in your script gets synthesized separately, then each chunk is time-stretched (WSOLA, via audiotsm) or padded to match the duration of the corresponding cue in the reference, and the original silence gaps are preserved. That's why the output lines up with the source material instead of just sounding vaguely like the speaker.

The inputs that matter

The full node has a lot of knobs, but a beginner sets maybe six of them:

• text (SRT) - what you want spoken. Wire this to a LoadSRT node.
• prompt_audio (AUDIO) - the reference clip. Feed it from a LoadAudio node.
• prompt_text (SRT) - a transcript of the reference audio, with cues that mark each sentence. Its durations drive the timing.
• if_mutiple_speaker - (yes, it's misspelled in the code, that's the name) toggles multi-speaker mode, where each subtitle is labeled SPK<letter> and the node builds separate voices per speaker.
• text2semantic_type - medium or large; medium is the default and is enough for most jobs.
• hf_token - a Hugging Face token with access to the gated fishaudio/fish-speech-1 repo (you must accept its conditions on the website first).

The rest - num_samples, max_new_tokens, top_p, repetition_penalty, temperature, seed, max_length, chunk_length - are the usual decoding controls. half runs in fp16 (less VRAM, slightly faster), compile enables torch.compile (faster after a slow first pass), and iterative_prompt is fish-speech's iterative decoding, on by default. The single output is AUDIO, a wav saved to ComfyUI/output/fish_speech/ - run it through the pack's PreViewAudio node to hear it.

Installing it

ComfyUI Manager has it - search "ComfyUI-FishSpeech" - or do it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/AIFSH/ComfyUI-FishSpeech
cd ComfyUI-FishSpeech
pip install -r requirements.txt

Then restart ComfyUI. Two things the README insists on: ffmpeg must be on your PATH, and the model weights (a ~1GB text2semantic checkpoint plus the codec and tokenizer) auto-download from Hugging Face on first run - which is exactly when the missing or wrong hf_token bites.

Where people get burned

• The gated repo. The weights live behind a "you must accept the conditions" gate on Hugging Face. No token, no download - you'll hit a 401 and the node sits there.
• samplerate won't build - a cmake error during pip install. The README's fix: pip -q install git+https://github.com/tuxu/python-samplerate.git@fix_cmake_dep.
• cannot import name 'weight_norm' from torch - the vendored code is old, so update torch.
• Timing drift in multi-speaker mode is fiddly and the per-speaker audio is exported with fixed SPK names, which can collide if you run twice. Keep one reference clip per voice.

One honest caveat: this pack is pinned to Fish Speech 1.4. The fishaudio team has shipped 1.5 and S2 since, so don't come here expecting the newest model - but 1.4 is still solid, and for timed dubbing in ComfyUI there isn't a lot that does this exact trick.