FireRedAudio 零样本声音克隆 · T8star-Aix

The pitch is short: drop in a five-second clip of someone talking, type what you want said, and out comes that person's voice saying it. No training, no fine-tune, no API key, no ElevenLabs bill. That's what the FireRedAudio TTS node does, and it's the node most people install this pack for in the first place.

It's one of 45 nodes in the comfyui-fireredaudio-T8 pack by T8star-Aix, a ComfyUI port of Xiaohongshu's FireRedAudio model (same lab that made FireRed-Image-Edit). FireRedAudio is a full audio language model, not just a text-to-speech wrapper - it does ASR, audio understanding and speech editing too. This node is the zero-shot voice cloning corner of it, and it's genuinely good at Chinese and English.

How it actually works

FireRedAudio is a Qwen-backbone audio LM paired with two extra pieces: a RedAE audio codec that turns a reference clip into conditioning tokens, and a flow-matching decoder that turns generated tokens back into a 24 kHz waveform. The reference clip gets encoded by the RedAE encoder and patch encoder, the model reads your target text, and it generates speech conditioned on that voice.

The clever bit for batch work: if you run several sentences with the same reference audio, the worker caches the RedAE/Patch Encoder conditions in-process, so it doesn't re-encode the voice for every sentence. The cache is keyed on file path, size and mtime, so it invalidates cleanly when you swap the reference.

The inputs that matter

Only three of these are you, really:

• model - wire this from the "FireRedAudio 模型/隔离运行时" loader node.
• prompt_audio - your reference clip. Anything from a few seconds up to a minute works; the pack's own reference-candidate tooling targets 3–15 second segments, and that's the sweet spot.
• target_text - the line to speak. This one is required; the node refuses to run with it empty.

The rest are worth knowing once. prompt_text is the transcript of the reference clip. Leave it blank and auto_transcribe_reference (on by default) will ASR the clip for you before generating - handy, but ASR can mangle a transcript, which is why the node hands back the actual transcript it used as an output. If you turn auto-ASR off, an empty transcript is a hard error. language is a simple zh/en toggle. The optional settings input comes from the "FireRedAudio 生成参数" node - that's where seed, quality preset (fast/balanced/high_quality), diffusion steps and CFG live.

Outputs: 生成音频 (AUDIO, wire it into the pack's "保存音频" node or any audio sink), a 运行报告 with generation stats, and 实际参考逐字稿 - worth a glance the first time, because it tells you exactly what the model "heard."

Installing it

Manager may not find it yet - the pack is awaiting registry review - so the reliable route is:

cd ComfyUI\custom_nodes
git clone https://github.com/T8mars/comfyui-fireredaudio-T8.git
cd comfyui-fireredaudio-T8
python scripts\setup_runtime.py

That last step matters more than it looks. This pack refuses to install its dependencies into ComfyUI's Python - requirements.txt is deliberately empty - and instead builds an isolated Python 3.10 / PyTorch 2.8 / Transformers 5.8 worker environment under the node folder. That's a real feature: it's the exact "my TTS upgrade broke all my other nodes" problem the audio community keeps hitting, solved by just not sharing the environment. But skip the setup step and the node won't run.

Then grab the model, the recommended int8-wo-safe-v1 (INT8-quantized, saves ~6 GB vs BF16):

python scripts\download_models.py --target "D:\ComfyUI\models\TTS\FireRedAudio" --profile full --variant int8-wo-safe-v1

Use --profile full here. TTS is generation, and generation needs the RedAE decoder that comes with the full profile; lite only covers ASR and understanding.

Where people get burned

• First run is slow. Cold start loads the whole model into the worker - the author's own logs show 60–120 seconds of model loading before the first WAV. Warm runs are the real speed, around 15 s on a 5090 laptop.
• It wants VRAM. TTS generation peaks around 20 GB of CUDA allocation. The loader's auto memory mode will pick sequential offload when you have less than ~36 GB free, so an 8–12 GB card can run it, just with decoder swapping.
• It's 24 kHz mono. Fine for dialogue and narration; don't expect a music-stem multi-channel thing.

One more thing, and it's the actual legal point: this is zero-shot cloning of a real person's voice. Only use it on voices you have the right to use. The README is explicit, and so am I.