WavTTS nodes for ComfyUI - zero-shot text-to-speech with reference-audio prompting, ComfyUI AUDIO wiring, optional Whisper transcription, local model storage, conservative dependency installation, and Aimdo/VRAM visualization support.

中文说明

Important: WavTTS requires a transcript for the reference audio. Whisper is optional, but reference_text is not.

Features

WavTTS 16 kHz inference - Loads the official WavTTS architecture through the upstream package while avoiding training-only imports.
ComfyUI AUDIO in/out - Reference audio is a ComfyUI AUDIO input and generated speech is a ComfyUI AUDIO output.
Bundled config assets - The nodepack includes assets/WavTTS.yaml, assets/config.json, and assets/vocab.txt.
Local model storage - WavTTS weights are stored under ComfyUI/models/wavtts/.
Hosted presets - FP32 and mixed BF16 presets are cataloged from drbaph/WavTTS.
Whisper helper node - Optional transcription node for creating reference_text from reference audio.
AIMDO visibility - WavTTS and Whisper modules register with ComfyUI model management using real tensor residency.
Hard unload - The unload node removes the ComfyUI memory entry and releases WavTTS weights to meta tensors when possible.
Safe installer - install.py uses uv when available, falls back to pip, and installs packages with --no-deps.

Installation

Method 1: ComfyUI Manager

Search for WavTTS or WavTTS-ComfyUI in ComfyUI Manager and install it.

After installation, restart ComfyUI. If optional runtime packages are missing, run install.py manually with the same Python environment that starts ComfyUI.

Method 2: Manual Install

cd ComfyUI/custom_nodes
git clone https://github.com/Saganaki22/WavTTS-ComfyUI.git
cd WavTTS-ComfyUI
python install.py

For ComfyUI portable on Windows:

..\..\python_embeded\python.exe install.py

For a venv install on Windows:

..\..\venv\Scripts\python.exe install.py

For this local setup:

C:\Users\drbaph\Documents\ComfyUI\venv\Scripts\python.exe C:\Users\drbaph\Documents\ComfyUI\custom_nodes\WavTTS-ComfyUI\install.py

The requirements.txt file is a commented dependency reference only. It is intentionally inert so automated installers do not bypass install.py and resolve dependency chains on their own.

Why `--no-deps`?

The upstream WavTTS package declares training/evaluation dependencies and pins that are not safe to blindly resolve inside an existing ComfyUI environment, including bitsandbytes, datasets, wandb, torchcodec, numpy<2.0.0, and pydantic<=2.10.6.

install.py protects ComfyUI by:

installing WavTTS from GitHub with --no-deps;
installing runtime helpers one by one with --no-deps;
trying uv pip install first and falling back to pip install;
never managing torch, torchaudio, torchvision, pydantic, torchcodec, bitsandbytes, datasets, or wandb.

If you need to install a package manually, use the same pattern:

python -m pip install x-transformers>=1.31.14 --no-deps
python -m uv pip install torchdiffeq --no-deps

Nodes

<details> <summary>1. WavTTS Load Model - Load WavTTS weights with dtype and attention controls</summary>

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | checkpoint | COMBO | WavTTS 16k FP32 - drbaph/WavTTS (auto-download) | Cataloged WavTTS preset. Local files are not added to the dropdown. | | dtype | COMBO | auto | auto, bf16, fp32. auto follows the checkpoint. Mixed BF16 stays mixed under auto and bf16; fp32 casts the whole model to FP32. | | device | COMBO | auto | auto, cuda, xpu, cpu. auto uses ComfyUI's current torch device. | | attention | COMBO | auto | auto, sdpa, flash_attention, sageattention. | | download_if_missing | BOOLEAN | True | Download the selected cataloged preset from drbaph/WavTTS into ComfyUI/models/wavtts/ when missing. |

Output:

wavtts_model - WAVTTS_MODEL, connect to Generate.

</details> <details> <summary>2. WavTTS Generate - Generate speech from text and reference audio</summary>

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | wavtts_model | WAVTTS_MODEL | required | Output from Load Model. | | reference_audio | AUDIO | required | Clean reference voice prompt. Short clips around 5-12 seconds are recommended. | | text | STRING, multiline | example text | Text to synthesize. | | reference_text | STRING, multiline | empty | Transcript for reference_audio. Required. | | Steps | INT | 50 | WavTTS flow sampling steps. Higher can improve quality but is slower. | | CFG | FLOAT | 3.0 | Classifier-free guidance strength. | | speed | FLOAT | 1.0 | Speech pace multiplier used for duration estimation. | | timestep_mapping | COMBO | power | Sampler timestep schedule: power, uniform, or sway_sampling. | | timestep_power | FLOAT | 2.0 | Exponent for power timestep mapping. | | shift | FLOAT | 3.0 | Flow timestep shift. | | cross_fade_seconds | FLOAT | 0.00 | Crossfade used when long text is split into multiple chunks. | | fixed_total_duration_seconds | FLOAT | 0.0 | 0 lets WavTTS estimate duration. Positive values force total prompt plus generated duration. | | seed | INT | 0 | 0 is unseeded. Positive values make generation repeatable. |

Reference cleanup settings are intentionally internal: max reference length 12s, trim enabled, silence threshold -42 dB, and target RMS 0.1.

Output:

audio - ComfyUI AUDIO.

</details> <details> <summary>3. WavTTS Whisper Transcribe - AUDIO to reference transcript</summary>

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | audio | AUDIO | required | Reference audio to transcribe. | | model | COMBO | whisper-large-v3-turbo (auto-download) | Whisper model selection. Auto-download choices are stored under ComfyUI/models/audio_encoders/. | | dtype | COMBO | auto | auto, fp16, bf16, fp32. | | language | COMBO | auto | Optional language hint. | | task | COMBO | transcribe | transcribe keeps source language; translate outputs English. | | chunk_length_s | INT | 30 | Whisper chunk length in seconds. 0 disables chunking. | | download_if_missing | BOOLEAN | True | Download selected Whisper model if missing. |

Output:

transcript - STRING, connect to reference_text on Generate.

</details> <details> <summary>4. WavTTS Unload Model - Release WavTTS memory</summary>

| Parameter | Type | Description | |-----------|------|-------------| | wavtts_model | WAVTTS_MODEL | Loaded model bundle to unload. |

The node unregisters the WavTTS module from ComfyUI memory tracking, detaches it from the accelerator, clears runtime caches, and uses to_empty(device="meta") when available so a large CPU copy is not retained.

</details>

Model Storage

All paths are resolved from folder_paths.models_dir, so portable and normal ComfyUI installs use the same layout:

ComfyUI/models/
  wavtts/
    wavtts-fp32.safetensors
    wavtts-mixed-bf16.safetensors
  audio_encoders/
    openai_whisper-large-v3-turbo/
    openai_whisper-large-v3/
    openai_whisper-medium/
    openai_whisper-small/
    openai_whisper-tiny/

When download_if_missing is enabled, the model dropdown works like a catalog: selecting a hosted drbaph/WavTTS preset downloads that exact filename into ComfyUI/models/wavtts/.

Available WavTTS Presets

| Dropdown name | Source | Local filename | Notes | |---------------|--------|----------------|-------| | WavTTS 16k FP32 - drbaph/WavTTS (auto-download) | drbaph/WavTTS | wavtts-fp32.safetensors | Stable inference path. | | WavTTS 16k Mixed BF16 - drbaph/WavTTS (auto-download) | drbaph/WavTTS | wavtts-mixed-bf16.safetensors | Keeps sensitive tensors FP32 and attention/FFN matmul tensors BF16. | | WavTTS 16k Official (auto-download) | worstchan/WavTTS | model_1200000.pt | Original training-style checkpoint. Large file; used only if selected. |

Pure BF16, pure FP16, and mixed FP16 presets are intentionally not advertised. Local testing showed FP16 WavTTS inference can produce non-finite audio samples after generation.

VRAM Notes

| Model | Approx file size | Expected VRAM | Notes | |-------|------------------|---------------|-------| | wavtts-fp32.safetensors | ~2.5 GiB | Higher than mixed models | Stable path, but heavier during generation. | | wavtts-mixed-bf16.safetensors | ~1.7 GiB | ~4-8 GB | Recommended low-VRAM option. |

Actual VRAM depends on text length, reference length, attention backend, and other ComfyUI models loaded at the same time.

Dtype Behavior

| Selection | FP32 checkpoint | Mixed BF16 checkpoint | |-----------|-----------------|-----------------------| | auto | Loads FP32 | Preserves FP32+BF16 layout | | bf16 | Casts whole model BF16 | Preserves FP32+BF16 layout | | fp32 | Loads/casts FP32 | Casts whole model FP32 |

This is deliberate. Mixed safetensors are authored with a per-tensor precision plan, so the loader does not flatten them just because the dropdown says bf16. Use fp32 only when you explicitly want to force every floating tensor back to FP32.

Precision Notes

The original WavTTS checkpoint is FP32. The clean FP32 safetensor is much smaller than the original .pt because it stores only inference EMA weights, not optimizer/training state.

In local testing, pure BF16, pure FP16, and mixed FP16 inference can sound bad or produce non-finite samples, including when the original FP32 weights are manually cast down. This appears to be a numerical stability issue in WavTTS's flow/ODE direct-waveform path, not a safetensors conversion issue. BF16 has the same two-byte storage class as FP16 but a much larger exponent range, so mixed BF16 is the supported low-VRAM preset.

Recommended choices:

For reliability: wavtts-fp32.safetensors with dtype=auto or dtype=fp32.
For FlashAttention: wavtts-mixed-bf16.safetensors with dtype=auto and attention=flash_attention.
For troubleshooting artifacts: go back to FP32 first.

Attention Backends

| Option | Behavior | |--------|----------| | auto | Resolves to sdpa. | | sdpa | Uses upstream WavTTS attn_backend="torch", which routes to PyTorch scaled-dot-product attention. | | flash_attention | Uses upstream WavTTS attn_backend="flash_attn". Requires flash_attn to be installed and low-precision tensors for the relevant attention projections. | | sageattention | Keeps upstream WavTTS on attn_backend="torch" and applies a runtime SageAttention SDPA patch when query/key/value tensors are CUDA FP16/BF16. |

FlashAttention and SageAttention are optional. Do not install them unless your Torch/CUDA environment already supports them.

Text Chunking

Long text is chunked with the same sentence split and UTF-8 byte-counting behavior as upstream WavTTS chunk_text. The max chunk length is estimated from reference text length, reference audio length, and speed, matching the upstream inference formula.

Because this is a ComfyUI node, reference audio enters as an AUDIO tensor instead of a file path. Reference trimming and resampling are implemented locally for ComfyUI.

Whisper

You do not need Whisper if you already know the exact transcript for your reference audio. Use WavTTS Whisper Transcribe when you want the workflow to produce reference_text automatically.

Whisper models are stored under ComfyUI/models/audio_encoders/ and are registered with ComfyUI model management after the Whisper node loads them, so Aimdo/VRAM tools can show their real tensor residency too.

AIMDO / MemoryVisualization

This wrapper registers WavTTS and Whisper torch modules through ComfyUI's current_loaded_models, so ComfyUI-MemoryVisualization can display them.

The implementation mirrors the local MisoTTS-ComfyUI approach: a custom Comfy patcher exposes an AIMDO-style VBAR view backed by the module's real tensors and actual device residency. No fake tensors are created.

Changing the selected checkpoint, dtype, device, or attention backend unloads the active WavTTS bundle before loading the new one.

Troubleshooting

<details> <summary>Generated audio is garbled</summary>

Use the FP32 checkpoint first:

checkpoint = WavTTS 16k FP32 - drbaph/WavTTS (auto-download)
dtype = auto
attention = auto

Pure BF16/FP16 is not recommended for WavTTS. Mixed BF16 is the supported low-VRAM path; mixed FP16 is unsupported because it produced non-finite audio in testing.

</details> <details> <summary>FlashAttention does not work</summary>

Use the mixed BF16 checkpoint and select attention=flash_attention. The upstream flash path requires the flash_attn package and CUDA-compatible low-precision attention tensors.

If your environment does not already support FlashAttention, leave attention=auto or attention=sdpa.

</details> <details> <summary>Whisper model is not visible in VRAM tools</summary>

Run the Whisper Transcribe node once. Whisper registers with ComfyUI model management when the pipeline first loads.

</details> <details> <summary>Model download fails</summary>

Disable download_if_missing and place files manually:

ComfyUI/models/wavtts/wavtts-fp32.safetensors
ComfyUI/models/wavtts/wavtts-mixed-bf16.safetensors

For Whisper, place local snapshots under:

ComfyUI/models/audio_encoders/

</details> <details> <summary>Installer tries to change too much</summary>

Do not run pip install -r requirements.txt; it is a commented reference only. Run:

python install.py

The installer uses uv-or-pip with --no-deps. If a package must be installed manually, also pass --no-deps.

</details>

Credits

WavTTS model and official code - cwx-worst-one/WavTTS.
WavTTS hosted safetensors - drbaph/WavTTS.
ComfyUI nodepack - Saganaki22/WavTTS-ComfyUI.
Whisper ASR - OpenAI Whisper checkpoints on Hugging Face.

License

This custom nodepack follows the repository license. WavTTS code, WavTTS model weights, Whisper models, FlashAttention, SageAttention, and other upstream packages each have their own licenses. Check the linked upstream repositories and model cards before redistribution or commercial use.

WavTTS

Nodes (4)

WavTTS-ComfyUI

Features

Installation

Method 1: ComfyUI Manager

Method 2: Manual Install

Why --no-deps?

Nodes

Model Storage

Available WavTTS Presets

VRAM Notes

Dtype Behavior

Precision Notes

Attention Backends

Text Chunking

Whisper

AIMDO / MemoryVisualization

Troubleshooting

Credits

License

Why `--no-deps`?