T5Gemma-TTS Generate

The name is a lie, in the best way: this node doesn't call any API and needs no key. T5GemmaTTS_Generate turns text into speech - English, Japanese, Chinese - entirely on your own GPU, with zero-shot voice cloning from a few seconds of reference audio, all inside a normal ComfyUI workflow. You wire a loader into it, type a sentence, and get a standard AUDIO tensor you can feed straight into SaveAudio or PreviewAudio. No round-trip through a web service, no audio files dragged in by hand.

It's the synthesize half of the small comfyui-t5gemma-tts pack by negaga53, a wrapper around Aratako's T5Gemma-TTS, a 2B encoder-decoder LLM built on Google's T5Gemma. Where the well-known open TTS models each occupy a corner - Kokoro is the tiny CPU-friendly one, Chatterbox (Llama-backbone, 0.5B) the ElevenLabs-competitor with cloning - T5Gemma-TTS is the multilingual entry. The KB's audio essay is blunt that reliable multi-language open TTS is still the weak spot next to the APIs; this is one of the honest attempts to close it, and it does EN/JA/ZH natively, which most open models simply don't.

How it works

Generation is a two-stage pipeline. First, T5Gemma generates a sequence of audio tokens with a standard LLM sampling loop - which is why top_k, top_p, temperature and seed are sitting right there in the inputs. Then the XCodec2 neural codec decodes those tokens into an actual waveform. The result is a mono audio tensor plus sample rate, in the ComfyUI AUDIO dict format, so every built-in audio node downstream just works.

Voice cloning is the part people actually want. Feed a reference clip into reference_audio and either type its transcript into reference_text or leave it blank - Whisper auto-transcribes it for you. The source is explicit about one trap here: if you supply reference_text but no reference_audio, it logs a warning and ignores the text entirely, so don't expect a text-only clone to happen.

The inputs that matter

• target_text - what to say. Multiline.
• language - auto (default), en, ja, zh. Handy if the auto-detector is wrong; setting it explicitly also tunes the duration estimate.
• reference_audio / reference_text - leave both empty for a plain (non-cloned) voice; add both for cloning.
• target_duration - seconds, 0 = auto-estimate from the text via phoneme-aware counting (g2p_en, pypinyin, pyopenjtalk). Explicit control is this model's party trick - set 5.0 and it lands near five seconds.
• The sampling trio (top_k 30, top_p 0.9, temperature 0.8) behaves exactly like an LLM: lower temperature for more consistent reads, higher for livelier but riskier ones. seed makes a given prompt reproducible.

Output: one audio (AUDIO) - wire it into SaveAudio to write a file, or PreviewAudio to listen without saving.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/negaga53/comfyui-t5gemma-tts
cd comfyui-t5gemma-tts
pip install -r requirements.txt
pip install https://huggingface.co/NandemoGHS/Anime-XCodec2-44.1kHz-v2/resolve/main/xcodec2-0.1.7.tar.gz
# optional, for proper Japanese phonemes:
pip install pyopenjtalk-plus[onnxruntime]

Restart ComfyUI. ComfyUI Manager also has it (search "T5Gemma-TTS"). Full VRAM detail lives in the loader article; budget ~10.6 GB for the default 2B model on an 8-bit encoder variant if you're tight.

Common issues

• "401 Unauthorized" on first run. The base model repo on HuggingFace is gated - you must log in and accept the license agreement on its page before the loader can download it. Do that once, then re-run.
• ImportError: xcodec2 module not found at load. You skipped the XCodec2 install line above; it's required, not optional.
• Whisper auto-transcription is slow on first use - it downloads a model and runs on GPU by default. If VRAM is tight, the loader's cpu_whisper flag moves it off the card.
• Windows quirks. Aratako's own README warns native Windows inference can hang or be inconsistent; WSL2 or Docker is the workaround.
• The license is real. The wrapper code is MIT, but the model weights are CC-BY-NC-4.0 + Gemma Terms of Use. Non-commercial only - check before you build a product on it.

One aside: this pack ships just the loader and this node. The README also describes a "T5Gemma-TTS Reference Audio" node, but it isn't in the code - don't go hunting for it; ComfyUI's built-in Load Audio plugs into reference_audio directly.