# WavTTS-ComfyUI **WavTTS nodes for ComfyUI** - zero-shot text-to-speech with reference-audio prompting, ComfyUI `AUDIO` wiring, optional Whisper transcription, local model storage, conservative dependency installation, and Aimdo/VRAM visualization support. [![Version](https://img.shields.io/badge/version-0.1.0-blue)](https://github.com/Saganaki22/WavTTS-ComfyUI) [![ComfyUI](https://img.shields.io/badge/ComfyUI-Custom%20Node-orange)](https://github.com/comfyanonymous/ComfyUI) [![WavTTS Models](https://img.shields.io/badge/HuggingFace-drbaph%2FWavTTS-blue)](https://huggingface.co/drbaph/WavTTS) [![Official Code](https://img.shields.io/badge/GitHub-cwx--worst--one%2FWavTTS-black)](https://github.com/cwx-worst-one/WavTTS) [中文说明](README_ZH.md) > Important: WavTTS requires a transcript for the reference audio. Whisper is optional, but `reference_text` is not. Screenshot 2026-06-04 015737

## Features - **WavTTS 16 kHz inference** - Loads the official WavTTS architecture through the upstream package while avoiding training-only imports. - **ComfyUI AUDIO in/out** - Reference audio is a ComfyUI `AUDIO` input and generated speech is a ComfyUI `AUDIO` output. - **Bundled config assets** - The nodepack includes `assets/WavTTS.yaml`, `assets/config.json`, and `assets/vocab.txt`. - **Local model storage** - WavTTS weights are stored under `ComfyUI/models/wavtts/`. - **Hosted presets** - FP32 and mixed BF16 presets are cataloged from [drbaph/WavTTS](https://huggingface.co/drbaph/WavTTS). - **Whisper helper node** - Optional transcription node for creating `reference_text` from reference audio. - **AIMDO visibility** - WavTTS and Whisper modules register with ComfyUI model management using real tensor residency. - **Hard unload** - The unload node removes the ComfyUI memory entry and releases WavTTS weights to meta tensors when possible. - **Safe installer** - `install.py` uses uv when available, falls back to pip, and installs packages with `--no-deps`. ## Installation ### Method 1: ComfyUI Manager Search for **WavTTS** or **WavTTS-ComfyUI** in ComfyUI Manager and install it. After installation, restart ComfyUI. If optional runtime packages are missing, run `install.py` manually with the same Python environment that starts ComfyUI. ### Method 2: Manual Install ```bash cd ComfyUI/custom_nodes git clone https://github.com/Saganaki22/WavTTS-ComfyUI.git cd WavTTS-ComfyUI python install.py ``` For ComfyUI portable on Windows: ```bat ..\..\python_embeded\python.exe install.py ``` For a venv install on Windows: ```bat ..\..\venv\Scripts\python.exe install.py ``` For this local setup: ```powershell C:\Users\drbaph\Documents\ComfyUI\venv\Scripts\python.exe C:\Users\drbaph\Documents\ComfyUI\custom_nodes\WavTTS-ComfyUI\install.py ``` The `requirements.txt` file is a commented dependency reference only. It is intentionally inert so automated installers do not bypass `install.py` and resolve dependency chains on their own. ### Why `--no-deps`? The upstream WavTTS package declares training/evaluation dependencies and pins that are not safe to blindly resolve inside an existing ComfyUI environment, including `bitsandbytes`, `datasets`, `wandb`, `torchcodec`, `numpy<2.0.0`, and `pydantic<=2.10.6`. `install.py` protects ComfyUI by: - installing WavTTS from GitHub with `--no-deps`; - installing runtime helpers one by one with `--no-deps`; - trying `uv pip install` first and falling back to `pip install`; - never managing `torch`, `torchaudio`, `torchvision`, `pydantic`, `torchcodec`, `bitsandbytes`, `datasets`, or `wandb`. If you need to install a package manually, use the same pattern: ```bash python -m pip install x-transformers>=1.31.14 --no-deps python -m uv pip install torchdiffeq --no-deps ``` ## Nodes

1. WavTTS Load Model - Load WavTTS weights with dtype and attention controls

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `checkpoint` | COMBO | `WavTTS 16k FP32 - drbaph/WavTTS (auto-download)` | Cataloged WavTTS preset. Local files are not added to the dropdown. | | `dtype` | COMBO | `auto` | `auto`, `bf16`, `fp32`. `auto` follows the checkpoint. Mixed BF16 stays mixed under `auto` and `bf16`; `fp32` casts the whole model to FP32. | | `device` | COMBO | `auto` | `auto`, `cuda`, `xpu`, `cpu`. `auto` uses ComfyUI's current torch device. | | `attention` | COMBO | `auto` | `auto`, `sdpa`, `flash_attention`, `sageattention`. | | `download_if_missing` | BOOLEAN | `True` | Download the selected cataloged preset from `drbaph/WavTTS` into `ComfyUI/models/wavtts/` when missing. | **Output:** - `wavtts_model` - `WAVTTS_MODEL`, connect to Generate.

2. WavTTS Generate - Generate speech from text and reference audio

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `wavtts_model` | WAVTTS_MODEL | required | Output from Load Model. | | `reference_audio` | AUDIO | required | Clean reference voice prompt. Short clips around 5-12 seconds are recommended. | | `text` | STRING, multiline | example text | Text to synthesize. | | `reference_text` | STRING, multiline | empty | Transcript for `reference_audio`. Required. | | `Steps` | INT | `50` | WavTTS flow sampling steps. Higher can improve quality but is slower. | | `CFG` | FLOAT | `3.0` | Classifier-free guidance strength. | | `speed` | FLOAT | `1.0` | Speech pace multiplier used for duration estimation. | | `timestep_mapping` | COMBO | `power` | Sampler timestep schedule: `power`, `uniform`, or `sway_sampling`. | | `timestep_power` | FLOAT | `2.0` | Exponent for `power` timestep mapping. | | `shift` | FLOAT | `3.0` | Flow timestep shift. | | `cross_fade_seconds` | FLOAT | `0.00` | Crossfade used when long text is split into multiple chunks. | | `fixed_total_duration_seconds` | FLOAT | `0.0` | `0` lets WavTTS estimate duration. Positive values force total prompt plus generated duration. | | `seed` | INT | `0` | `0` is unseeded. Positive values make generation repeatable. | Reference cleanup settings are intentionally internal: max reference length `12s`, trim enabled, silence threshold `-42 dB`, and target RMS `0.1`. **Output:** - `audio` - ComfyUI `AUDIO`.

3. WavTTS Whisper Transcribe - AUDIO to reference transcript

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `audio` | AUDIO | required | Reference audio to transcribe. | | `model` | COMBO | `whisper-large-v3-turbo (auto-download)` | Whisper model selection. Auto-download choices are stored under `ComfyUI/models/audio_encoders/`. | | `dtype` | COMBO | `auto` | `auto`, `fp16`, `bf16`, `fp32`. | | `language` | COMBO | `auto` | Optional language hint. | | `task` | COMBO | `transcribe` | `transcribe` keeps source language; `translate` outputs English. | | `chunk_length_s` | INT | `30` | Whisper chunk length in seconds. `0` disables chunking. | | `download_if_missing` | BOOLEAN | `True` | Download selected Whisper model if missing. | **Output:** - `transcript` - `STRING`, connect to `reference_text` on Generate.

4. WavTTS Unload Model - Release WavTTS memory

| Parameter | Type | Description | |-----------|------|-------------| | `wavtts_model` | WAVTTS_MODEL | Loaded model bundle to unload. | The node unregisters the WavTTS module from ComfyUI memory tracking, detaches it from the accelerator, clears runtime caches, and uses `to_empty(device="meta")` when available so a large CPU copy is not retained.

## Model Storage All paths are resolved from `folder_paths.models_dir`, so portable and normal ComfyUI installs use the same layout: ```text ComfyUI/models/ wavtts/ wavtts-fp32.safetensors wavtts-mixed-bf16.safetensors audio_encoders/ openai_whisper-large-v3-turbo/ openai_whisper-large-v3/ openai_whisper-medium/ openai_whisper-small/ openai_whisper-tiny/ ``` When `download_if_missing` is enabled, the model dropdown works like a catalog: selecting a hosted `drbaph/WavTTS` preset downloads that exact filename into `ComfyUI/models/wavtts/`. ### Available WavTTS Presets | Dropdown name | Source | Local filename | Notes | |---------------|--------|----------------|-------| | `WavTTS 16k FP32 - drbaph/WavTTS (auto-download)` | [drbaph/WavTTS](https://huggingface.co/drbaph/WavTTS) | `wavtts-fp32.safetensors` | Stable inference path. | | `WavTTS 16k Mixed BF16 - drbaph/WavTTS (auto-download)` | [drbaph/WavTTS](https://huggingface.co/drbaph/WavTTS) | `wavtts-mixed-bf16.safetensors` | Keeps sensitive tensors FP32 and attention/FFN matmul tensors BF16. | | `WavTTS 16k Official (auto-download)` | [worstchan/WavTTS](https://huggingface.co/worstchan/WavTTS) | `model_1200000.pt` | Original training-style checkpoint. Large file; used only if selected. | Pure BF16, pure FP16, and mixed FP16 presets are intentionally not advertised. Local testing showed FP16 WavTTS inference can produce non-finite audio samples after generation. ## VRAM Notes | Model | Approx file size | Expected VRAM | Notes | |-------|------------------|---------------|-------| | `wavtts-fp32.safetensors` | ~2.5 GiB | Higher than mixed models | Stable path, but heavier during generation. | | `wavtts-mixed-bf16.safetensors` | ~1.7 GiB | ~4-8 GB | Recommended low-VRAM option. | Actual VRAM depends on text length, reference length, attention backend, and other ComfyUI models loaded at the same time. ## Dtype Behavior | Selection | FP32 checkpoint | Mixed BF16 checkpoint | |-----------|-----------------|-----------------------| | `auto` | Loads FP32 | Preserves FP32+BF16 layout | | `bf16` | Casts whole model BF16 | Preserves FP32+BF16 layout | | `fp32` | Loads/casts FP32 | Casts whole model FP32 | This is deliberate. Mixed safetensors are authored with a per-tensor precision plan, so the loader does not flatten them just because the dropdown says `bf16`. Use `fp32` only when you explicitly want to force every floating tensor back to FP32. ## Precision Notes The original WavTTS checkpoint is FP32. The clean FP32 safetensor is much smaller than the original `.pt` because it stores only inference EMA weights, not optimizer/training state. In local testing, pure BF16, pure FP16, and mixed FP16 inference can sound bad or produce non-finite samples, including when the original FP32 weights are manually cast down. This appears to be a numerical stability issue in WavTTS's flow/ODE direct-waveform path, not a safetensors conversion issue. BF16 has the same two-byte storage class as FP16 but a much larger exponent range, so mixed BF16 is the supported low-VRAM preset. Recommended choices: - For reliability: `wavtts-fp32.safetensors` with `dtype=auto` or `dtype=fp32`. - For FlashAttention: `wavtts-mixed-bf16.safetensors` with `dtype=auto` and `attention=flash_attention`. - For troubleshooting artifacts: go back to FP32 first. ## Attention Backends | Option | Behavior | |--------|----------| | `auto` | Resolves to `sdpa`. | | `sdpa` | Uses upstream WavTTS `attn_backend="torch"`, which routes to PyTorch scaled-dot-product attention. | | `flash_attention` | Uses upstream WavTTS `attn_backend="flash_attn"`. Requires `flash_attn` to be installed and low-precision tensors for the relevant attention projections. | | `sageattention` | Keeps upstream WavTTS on `attn_backend="torch"` and applies a runtime SageAttention SDPA patch when query/key/value tensors are CUDA FP16/BF16. | FlashAttention and SageAttention are optional. Do not install them unless your Torch/CUDA environment already supports them. ## Text Chunking Long text is chunked with the same sentence split and UTF-8 byte-counting behavior as upstream WavTTS `chunk_text`. The max chunk length is estimated from reference text length, reference audio length, and `speed`, matching the upstream inference formula. Because this is a ComfyUI node, reference audio enters as an `AUDIO` tensor instead of a file path. Reference trimming and resampling are implemented locally for ComfyUI. ## Whisper You do not need Whisper if you already know the exact transcript for your reference audio. Use **WavTTS Whisper Transcribe** when you want the workflow to produce `reference_text` automatically. Whisper models are stored under `ComfyUI/models/audio_encoders/` and are registered with ComfyUI model management after the Whisper node loads them, so Aimdo/VRAM tools can show their real tensor residency too. ## AIMDO / MemoryVisualization This wrapper registers WavTTS and Whisper torch modules through ComfyUI's `current_loaded_models`, so [ComfyUI-MemoryVisualization](../ComfyUI-MemoryVisualization) can display them. The implementation mirrors the local MisoTTS-ComfyUI approach: a custom Comfy patcher exposes an AIMDO-style VBAR view backed by the module's real tensors and actual device residency. No fake tensors are created. Changing the selected checkpoint, dtype, device, or attention backend unloads the active WavTTS bundle before loading the new one. ## Troubleshooting

Generated audio is garbled

Use the FP32 checkpoint first: ```text checkpoint = WavTTS 16k FP32 - drbaph/WavTTS (auto-download) dtype = auto attention = auto ``` Pure BF16/FP16 is not recommended for WavTTS. Mixed BF16 is the supported low-VRAM path; mixed FP16 is unsupported because it produced non-finite audio in testing.

FlashAttention does not work

Use the mixed BF16 checkpoint and select `attention=flash_attention`. The upstream flash path requires the `flash_attn` package and CUDA-compatible low-precision attention tensors. If your environment does not already support FlashAttention, leave `attention=auto` or `attention=sdpa`.

Whisper model is not visible in VRAM tools

Run the Whisper Transcribe node once. Whisper registers with ComfyUI model management when the pipeline first loads.

Model download fails

Disable `download_if_missing` and place files manually: ```text ComfyUI/models/wavtts/wavtts-fp32.safetensors ComfyUI/models/wavtts/wavtts-mixed-bf16.safetensors ``` For Whisper, place local snapshots under: ```text ComfyUI/models/audio_encoders/ ```

Installer tries to change too much

Do not run `pip install -r requirements.txt`; it is a commented reference only. Run: ```bash python install.py ``` The installer uses uv-or-pip with `--no-deps`. If a package must be installed manually, also pass `--no-deps`.

## Credits - **WavTTS model and official code** - [cwx-worst-one/WavTTS](https://github.com/cwx-worst-one/WavTTS). - **WavTTS hosted safetensors** - [drbaph/WavTTS](https://huggingface.co/drbaph/WavTTS). - **ComfyUI nodepack** - [Saganaki22/WavTTS-ComfyUI](https://github.com/Saganaki22/WavTTS-ComfyUI). - **Whisper ASR** - OpenAI Whisper checkpoints on Hugging Face. ## License This custom nodepack follows the repository license. WavTTS code, WavTTS model weights, Whisper models, FlashAttention, SageAttention, and other upstream packages each have their own licenses. Check the linked upstream repositories and model cards before redistribution or commercial use.

WavTTS Unload Model

Inputs (1)

Outputs (1)