comfyui-t5gemma-tts

ComfyUI – T5Gemma-TTS

ComfyUI custom nodes for T5Gemma-TTS, a multilingual text-to-speech model with voice cloning and duration control, based on the T5Gemma encoder-decoder LLM architecture.

Features

• Multilingual TTS — English, Chinese, and Japanese
• Voice Cloning — Zero-shot voice cloning from reference audio
• Duration Control — Explicit control over generated audio length (auto-estimation when not specified)
• ComfyUI Native — Outputs standard AUDIO type, compatible with built-in SaveAudio and PreviewAudio nodes

Nodes

T5Gemma-TTS Model Loader

Loads the model and tokenizers from a HuggingFace repo ID or local path.

| Parameter | Default | Description | |---|---|---| | model_dir | Aratako/T5Gemma-TTS-2b-2b | HF repo ID or local path | | device | auto | auto, cuda, or cpu | | xcodec2_model_name | (blank) | Override XCodec2 model name | | use_torch_compile | False | Enable torch.compile (CUDA only) | | cpu_codec | False | Run XCodec2 on CPU (saves ~3.5 GB VRAM) | | cpu_whisper | False | Run Whisper on CPU (saves ~5 GB VRAM) |

T5Gemma-TTS Generate

Generates speech audio from text.

| Parameter | Default | Description | |---|---|---| | model | — | Model from the loader node | | target_text | — | Text to synthesize | | top_k | 30 | Top-k sampling | | top_p | 0.9 | Nucleus sampling | | temperature | 0.8 | Sampling temperature | | seed | 1 | Random seed | | reference_audio | (optional) | Reference audio for voice cloning | | reference_text | (optional) | Transcript of reference (auto-transcribed if blank) | | target_duration | 0.0 | Duration in seconds (0 = auto) | | language | auto | auto, en, ja, zh |

T5Gemma-TTS Reference Audio

Loads a reference audio file from disk for voice cloning.

| Parameter | Description | |---|---| | audio_path | Path to a WAV/FLAC/MP3 file |

Installation

1. Clone or copy this folder into your ComfyUI custom_nodes/ directory.

2. Install dependencies:

cd ComfyUI/custom_nodes/comfyui-t5gemma-tts
pip install -r requirements.txt

3. Install XCodec2 (if not already installed):

# Anime-XCodec2 variant (44.1 kHz, recommended for Japanese voices)
pip install https://huggingface.co/NandemoGHS/Anime-XCodec2-44.1kHz-v2/resolve/main/xcodec2-0.1.7.tar.gz

# OR original XCodec2 (16 kHz, recommended for English/Chinese)
# pip install xcodec2==0.1.5 --no-deps

4. (Optional) Install Japanese phoneme support:

pip install pyopenjtalk-plus[onnxruntime]

5. Restart ComfyUI.

Workflow Example

[T5Gemma-TTS Model Loader] → model
                                ↓
[T5Gemma-TTS Generate] → AUDIO → [Save Audio]
        ↑ (optional)
[Load Audio / T5Gemma-TTS Reference Audio] → reference_audio

Basic TTS

1. Add T5Gemma-TTS Model Loader — set model_dir to Aratako/T5Gemma-TTS-2b-2b
2. Add T5Gemma-TTS Generate — connect model, type your text
3. Add Save Audio — connect the audio output
4. Run the workflow

Voice Cloning

1. Same as above, plus:
2. Add Load Audio (or T5Gemma-TTS Reference Audio) — load a reference WAV
3. Connect it to the reference_audio input of the Generate node
4. Optionally provide reference_text (otherwise Whisper will auto-transcribe)

VRAM Requirements

| Model | Approx. VRAM | |---|---| | T5Gemma-TTS-2b-2b | ~10.6 GB | | T5Gemma-TTS-2b-2b-encoder-8bit | ~8.6 GB | | T5Gemma-TTS-2b-2b-encoder-4bit | ~7.6 GB |

Use cpu_codec and/or cpu_whisper to reduce VRAM usage further.

Credits

• T5Gemma-TTS: Aratako/T5Gemma-TTS — MIT License
• XCodec2: HKUSTAudio/xcodec2
• XCodec2-Variant: NandemoGHS/Anime-XCodec2-44.1kHz-v2

License

MIT