ComfyUI custom nodes for Zyphra/ZONOS2, with text-to-speech, audio-only voice cloning, SDPA and FlashAttention inference, native progress reporting, and ComfyUI/AIMDO memory management.

简体中文

ZONOS2 is our latest text-to-speech model trained on more than 6 million hours of varied multilingual speech, delivering expressiveness and quality on par with—or even surpassing—top TTS providers at low latency with MoE. ZONOS2 excels at high-fidelity and naturalistic voice cloning.

During inference we use nemo TN normalized UTF-8 bytes and an ECAPA-TDNN embedding to generate DAC tokens with our MoE backbone. An inference overview can be seen below.

ZONOS2 inference overview

Language Support

| Tier | Languages | | --- | --- | | Tier 1 | English, Mandarin Chinese, Japanese | | Tier 2 | Korean, Russian, Italian, Portuguese, French, Spanish, Vietnamese, German, Hebrew, Dutch | | Tier 3 | Swedish, Hindi, Tamil, Telugu, Thai, Norwegian, Bengali, Tagalog, Arabic, Danish, Indonesian, Polish, Ukrainian, Romanian, Finnish, Hungarian, Lithuanian, Estonian, Slovak, Croatian, Latvian |

[!WARNING] Only clone voices you own or have explicit permission to use. Malicious impersonation, fraud, deception, harassment, abuse, evasion of consent, or any use intended to cause harm is strictly forbidden by this project's acceptable-use policy.

Highlights

Native in-process PyTorch implementation for Windows and Linux.
Standard ComfyUI AUDIO input and 44.1 kHz AUDIO output.
Audio-only zero-shot voice cloning; no reference transcript is required.
All released ZONOS2 speaking-rate and quality-conditioning buckets.
auto, bf16, and fp16 runtime dtypes.
Validated 9.78 GiB mixed FP8 checkpoint with FP8 E4M3 MoE expert gate/up weights and BF16-sensitive paths.
Automatic FlashAttention selection with PyTorch SDPA fallback.
Native ComfyUI progress bars and in-place CLI tqdm progress.
Real tensor registration with ComfyUI model management and AIMDO.
Full cleanup and reload when model, dtype, or attention settings change.

<details> <summary>Installation</summary>

ComfyUI Manager

Once listed in ComfyUI Manager, search for ZONOS2 TTS and install it normally.

To install it immediately through Manager:

Open Manager.
Choose Install via Git URL.
Enter https://github.com/Saganaki22/Zonos2_TTS-ComfyUI.
Restart ComfyUI.

Manual installation

From the ComfyUI/custom_nodes directory:

git clone https://github.com/Saganaki22/Zonos2_TTS-ComfyUI.git
..\venv\Scripts\python.exe Zonos2_TTS-ComfyUI\install.py

Linux portable or venv installations can run:

git clone https://github.com/Saganaki22/Zonos2_TTS-ComfyUI.git
../venv/bin/python Zonos2_TTS-ComfyUI/install.py

For a uv-managed ComfyUI environment:

uv run python Zonos2_TTS-ComfyUI/install.py

When uv is available, install.py automatically runs uv pip install --python <active ComfyUI Python>. Otherwise, it falls back to python -m pip. Restart ComfyUI after installing or updating. The helper installs only missing lightweight dependencies and does not replace ComfyUI's torch, torchaudio, or transformers.

</details> <details> <summary>Models and automatic downloads</summary>

Models are managed under:

ComfyUI/models/zonos2/
├── zonos2-bf16.safetensors
├── zonos2-fp8-mixed.safetensors
├── dac_44khz/
│   ├── .gitattributes
│   ├── config.json
│   ├── model.safetensors
│   ├── preprocessor_config.json
│   └── README.md
└── speaker_encoder/
    ├── config.json
    ├── configuration_ecapa_tdnn.py
    ├── feature_extraction_ecapa_tdnn.py
    ├── model.safetensors
    ├── modeling_ecapa_tdnn.py
    ├── preprocessor_config.json
    ├── tokenizer_config.json
    └── tokenizer_ecapa_tdnn.py

Two model presets are included:

ZONOS2 BF16 downloads zonos2-bf16.safetensors and missing shared assets from drbaph/ZONOS2-BF16.
ZONOS2 FP8 Mixed downloads zonos2-fp8-mixed.safetensors and missing shared assets from drbaph/ZONOS2-FP8.

Downloads are checked independently and reuse the same local folders across both presets:

The selected root checkpoint is downloaded whenever it is missing, even if dac_44khz/ and speaker_encoder/ already exist.
A complete local dac_44khz/ folder is reused without downloading it again.
A complete local speaker_encoder/ folder is reused without downloading it again.
The DAC is checked when the model bundle loads. The speaker encoder is checked lazily when voice cloning is first used.

Enable download_if_missing to allow only the missing pieces to download. Disable it for fully offline operation after placing every required asset in the paths above.

The custom node ships the official architecture configuration at assets/params.json. BF16 checkpoints are validated against the native architecture. Mixed FP8 checkpoints are validated against their format metadata, quantization policy, required ComfyUI quantization tensors, and runtime weight shapes before inference. Locally added .safetensors files also appear in the model dropdown.

The mixed FP8 policy is deliberately conservative:

FP8 E4M3: MoE expert gate/up (w13) projections.
BF16: attention, dense FFN, expert-down (w2), LM head, routers, embeddings, norms, speaker projections, biases, and temperatures.

This reduces the main checkpoint from about 14.28 GiB to 9.78 GiB while protecting the paths that failed under broader FP8 conversion. FP8 is primarily a memory-saving option and is not guaranteed to generate faster than BF16.

Upload the complete dac_44khz/ and speaker_encoder/ directories to the model repository rather than reconstructing them from the abbreviated tree. At runtime, the DAC directly requires config.json and model.safetensors. The speaker encoder directly requires config.json, model.safetensors, configuration_ecapa_tdnn.py, and modeling_ecapa_tdnn.py because it is loaded through Transformers remote-code support. The preprocessor, feature-extractor, tokenizer, README, and Git metadata files are not directly used by this node's current inference path, but retaining the complete upstream folders preserves a valid, reusable Hugging Face package.

</details> <details> <summary>Nodes and settings</summary>

ZONOS2 Model Loader

| Input | Default | Options | Description | |---|---|---|---| | model | ZONOS2 BF16 preset | BF16 preset, mixed FP8 preset, and local safetensors | Loads from ComfyUI/models/zonos2; each preset downloads from its listed Hugging Face repository. | | dtype | auto | auto, bf16, fp16 | auto preserves standard checkpoint dtype and automatically uses BF16 compute for mixed FP8. Mixed FP8 accepts auto or bf16; fp16 is rejected. | | attention | auto | auto, SDPA, flash_attention | auto uses FlashAttention when installed and compatible, otherwise SDPA. | | download_if_missing | true | boolean | Independently downloads only the missing selected checkpoint, DAC, or speaker encoder assets. Existing shared folders are reused. |

Changing the model, dtype, or attention backend intentionally hard-unloads the previous bundle before loading the replacement.

ZONOS2 Voice Generation

Accepts zonos2_model and UTF-8 text, then returns mono ComfyUI AUDIO at 44.1 kHz.

ZONOS2 Voice Clone

Accepts zonos2_model, UTF-8 text, and a required native ComfyUI reference_audio noodle. ZONOS2 extracts its official 2048-dimensional ECAPA-TDNN speaker embedding directly from the audio; reference text is not required. The model conditions on this fixed speaker embedding rather than the reference waveform or transcript, so voice identity can transfer more reliably than accent, cadence, emotion, and other utterance-level prosody.

| Clone setting | Default | Description | |---|---:|---| | reference_audio | required | Use 5–30 seconds of clear, single-speaker speech that demonstrates the desired accent. The node accepts at most 60 seconds and clips longer input with a CLI warning. | | clean_speaker_background | false | Matches the upstream default. Enable only for genuinely clean studio-like speech; leave disabled for ordinary recordings or audible room tone, noise, reverb, or ambience. | | accurate_mode | true | Requests stricter adherence to the speaker embedding. It may improve identity similarity, but it is not a dedicated accent or prosody control. Disable for looser, potentially more expressive conditioning. |

Sampling controls

| Control | Default | Range | Description | |---|---:|---:|---| | max_new_tokens | 1024 | 32–6000, step 8 | Maximum generated DAC-code frames. Higher values allow longer speech and use more time and KV-cache memory. | | temperature | 1.15 | 0–2 | Sampling randomness; 0 is greedy. | | top_k | 106 | 0–1026 | Keeps the K most likely tokens; 0 disables it. | | top_p | 0.0 | 0–1 | Nucleus sampling threshold; 0 disables it. | | min_p | 0.18 | 0–1 | Removes tokens below a fraction of the most likely token; 0 disables it. | | repetition_window | 50 | 0–512 | Recent frames checked for repetition; 0 disables tracking. | | repetition_penalty | 1.2 | 1–2 | Discourages repeated audio tokens; 1 disables the penalty. | | repetition_codebooks | 8 | -1–9 | Number of codebooks receiving the penalty; -1 means all and 0 means none. | | seed | 0 | 0–9223372036854775807 | Positive values make identical runs repeatable; 0 uses the current random state. |

ZONOS2 conditioning

Every conditioning dropdown includes default, which leaves that feature unconditioned.

| Control | Released buckets | Default | |---|---|---| | speaking_rate | 8 UTF-8 bytes-per-second ranges from 0–8 through 40+ | default | | loudness_lufs | 12 ranges from below -50 through -5+ LUFS | default | | estimated_snr | 12 ranges from below 0 through 60+ dB | default | | maximum_pause | 12 ranges from 0–0.5 through 5.5–6 seconds | default | | estimated_bandlimit_hz | 8 ranges from 495.3–3433 through 21062–24000 Hz | default | | leading_silence | 8 ranges from 0–0.05 through 4+ seconds | default | | trailing_silence | 8 ranges from 0–0.05 through 4+ seconds | 0.25–0.5 |

</details> <details> <summary>Reference audio guidance</summary>

Recommended: 5–30 seconds of clean, uninterrupted speech from one speaker.
Maximum accepted by this nodepack: 60 seconds.
Longer input is clipped to the first 60 seconds before resampling and embedding.
The ComfyUI CLI reports the supplied duration and clipping action.
Very short references below 5 seconds also produce a recommendation warning.
Avoid music, overlapping speakers, heavy denoising artifacts, strong reverb, and long silence.
Include clear examples of the desired accent and, where practical, use reference speech in the same language as the generated text.
Leave clean_speaker_background disabled unless the recording is genuinely clean. This flag describes the recording condition; it is not a denoiser.

The 60-second ceiling is a practical memory and latency guardrail in this integration, not a documented architectural limit of the upstream ZONOS2 model.

</details> <details> <summary>Transformers compatibility</summary>

The nodepack was tested end to end with Transformers 5.3.0. DAC decoding and speaker-encoder forward passes were also tested with:

5.0.0, 5.2.0, 5.3.0, 5.5.4, and 5.12.0

Supported and tested range: transformers>=5.0.0,<=5.12.0.

Transformers 4.x is not supported by this nodepack because common current ComfyUI environments conflict with the older NumPy and huggingface-hub constraints required by those releases. install.py reports whether the installed Transformers version is inside the tested range but never replaces it automatically.

</details> <details> <summary>Memory management and progress</summary>

The ZONOS2 model, DAC decoder, and lazy speaker encoder are registered as real PyTorch modules with ComfyUI model management.

The BF16 main model is estimated at approximately 14.324 GiB. The loader adds a 3 GiB runtime reserve, producing an automatic AIMDO cutoff of approximately 17.324 GiB total VRAM.
The mixed FP8 main model is approximately 9.78 GiB. Its equivalent automatic AIMDO cutoff is approximately 12.78 GiB total VRAM.
With dtype: auto, mixed FP8 keeps FP8 expert storage and selects BF16 compute from checkpoint metadata. Native FP8 execution requires current ComfyUI/comfy-kitchen support and compatible hardware; unsupported hardware uses ComfyUI's compatible dequantized fallback and may be slower.
When AIMDO DynamicVRAM is enabled, GPUs below that cutoff, including typical 8 GiB, 12 GiB, and 16 GiB cards, use ComfyUI's real CoreModelPatcher and AIMDO VBAR path.
On that dynamic path, the main model keeps its large MoE expert weights file-backed and pages only the selected experts into VRAM on demand. VBAR allocations, page faults, residency, and eviction are real AIMDO operations rather than visualization emulation.
GPUs with enough total VRAM for the estimated model plus the 3 GiB reserve use the static GPU path. This avoids retained CPU model backing and gives the direct CUDA expert path.
The automatic choice uses the GPU's total VRAM capacity, not its currently free VRAM. A 24 GiB or 32 GiB GPU therefore selects the static path even when other loaded models are using part of its VRAM; ComfyUI may unload those models to make room.
Shared attention, routing, embedding, and output weights remain resident on the dynamic path to avoid per-token paging overhead.
Each expert's FP8 w13 and BF16 w2 projections are independently AIMDO-pageable, so only projections belonging to routed experts need to become resident. They use separate VBAR allocations because their storage formats differ.
The smaller DAC and speaker encoder use ComfyUI's standard static patcher. Memory Visualization therefore shows them as orange static VRAM rows using their real loaded sizes.
Systems without AIMDO always use ComfyUI's standard static model patcher and load weights directly to the selected device, regardless of GPU capacity.
The main model's AIMDO bar reflects actual expert-page residency and updates as pages are faulted or evicted.
Dynamic paging necessarily keeps one file-backed CPU source for weights that may be evicted. These clean mapped pages are reclaimable by the operating system; the node does not create a second full expert copy.
Reusing identical loader settings resumes the existing bundle.
Changing model, dtype, or attention unregisters the old bundle, moves its tensors to meta, clears references, runs garbage collection, and empties accelerator caches before loading the replacement.
Loading and generation use native ComfyUI progress bars plus CLI tqdm.
Single-token MoE decoding dispatches only the selected expert weights instead of scanning every expert, substantially reducing autoregressive generation overhead without changing generated tokens.

Without DynamicVRAM, measured BF16 CUDA allocation with the main model and DAC loaded is approximately 14.7 GiB. With AIMDO VBAR, resident expert-weight memory adapts to available VRAM. The 3 GiB loader reserve is a path-selection allowance, not a guarantee that every workflow will fit; always leave additional VRAM for the KV cache, speaker encoder, DAC, ComfyUI, and other nodes.

</details> <details> <summary>Troubleshooting</summary>

The model dropdown download fails or returns 404

Confirm the selected repository contains its root checkpoint plus dac_44khz/ and speaker_encoder/: drbaph/ZONOS2-BF16 for BF16 or drbaph/ZONOS2-FP8 for mixed FP8. Partial uploads can fail until the requested file is available. Existing complete local asset folders are not downloaded again.

FP8 reports an unsupported format or a 3D linear weight

Update this custom node and fully restart ComfyUI so Python reloads the current FP8 format. Version 0.1.7 rejects the retired all-layer FP8 layout and malformed expert tensors before inference. The supported checkpoint metadata format is the expert-gate/up-only layout used by zonos2-fp8-mixed.safetensors.

FlashAttention is unavailable

Use attention: auto or SDPA. Auto falls back to SDPA when FlashAttention is not installed, the device is not CUDA, or the selected dtype is incompatible.

CUDA out of memory

Unload other large ComfyUI models, reduce max_new_tokens, use ComfyUI offloading, or restart ComfyUI after a failed allocation. ZONOS2 BF16 plus DAC uses roughly 14.7 GiB before generation cache growth. Automatic VBAR selection is based on total VRAM: GPUs below approximately 17.324 GiB use AIMDO VBAR when DynamicVRAM is enabled, while larger GPUs select the static path.

Voice cloning sounds weak or inaccurate

Use 5–30 seconds of clear, single-speaker speech with little silence. Set clean_speaker_background to match the actual recording instead of enabling it automatically, and leave accurate_mode enabled for stricter identity conditioning.

The voice matches, but the accent or cadence does not

This can be a model limitation rather than an integration fault. ZONOS2 receives one 2048-dimensional speaker embedding from the reference, not the reference waveform, audio tokens, or transcript. That embedding is designed primarily to represent speaker identity and can discard accent, rhythm, emotion, and other utterance-level variation. Use a reference that clearly demonstrates the target accent, preferably in the target language, and compare several fixed seeds. If sampling varies too much, trying temperature around 0.8–1.0 may make results more stable, but it cannot add accent information absent from the embedding.

accurate_mode enables the upstream accurate-cloning token; it is not an accent-strength control. The integration currently uses ZONOS2's supported raw UTF-8 text path. Upstream NeMo text normalization can improve written-number, date, and currency pronunciation, but it does not supply reference accent information.

The reference is longer than 60 seconds

The node clips it to the first 60 seconds and prints a warning such as: Reference audio is 75.00 seconds; the node accepts at most 60.0 seconds. Clipping to the first 60.0 seconds.

Output stops too early

Increase max_new_tokens. The model may still stop earlier when it emits end-of-audio.

Transformers import or model-loading errors

Check the startup output from install.py and use a release in the tested 5.0.0–5.12.0 range. Version 5.3.0 is the recommended baseline.

A model, dtype, or attention change reloads everything

This is expected. The old tensors are hard-unloaded to prevent stale AIMDO registrations and retained VRAM.

</details> <details> <summary>Example workflow</summary>

Load example_workflows/zonos2_tts_example_workflow.json in ComfyUI. It contains normal generation and voice-cloning branches. Select your own audio file in the LoadAudio node before running the clone branch.

</details> <details> <summary>Licenses and responsible use</summary>

The original ZONOS2 model weights are released under the Apache License 2.0.
This ComfyUI integration code is released under the MIT License.
DAC, speaker encoder, and other dependencies remain governed by their respective upstream licenses.

The Apache-2.0 model license and this project's acceptable-use policy are separate. Regardless of license permissions, this project must not be used for malicious impersonation, fraud, deception, harassment, non-consensual voice cloning, or causing harm.

</details>

Citation

If you find this model useful in an academic context, please cite:

@misc{zyphra2025zonos,
  title     = {Zonos V2 Technical Report},
  author    = {Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close, Beren Millidge},
  year      = {2026},
}

ZONOS2 TTS

Nodes (3)

ZONOS2 TTS ComfyUI