ComfyUI-PersonaPalette

ComfyUI-PersonaPalette

[en | ja]

Version: 0.1.1 License: GPL-3.0

PersonaPalette is a ComfyUI node set for creating character-voice references from existing audio.

It adjusts a source voice along five persona axes: Youth, Brightness, Energy, Softness, and Composure. The result is intended as reference audio for TTS, voice cloning, and reference-based speech synthesis workflows such as Irodori-TTS.

PersonaPalette is not a TTS engine and not a final mastering effect. It is a preparation step for reference-based TTS: take a source voice, nudge its character impression with DSP, then naturalize the result with Seed-VC V2.

Use it when your reference voice is close, but not quite the right character yet.

Example Workflow

The examples directory includes a small PersonaPalette workflow with the source audio and generated result.

| File | What it is | | --- | --- | | sample_workflow.json | Importable ComfyUI sample workflow | | F2_surprise_regular_33.wav | Source voice used by the sample workflow | | new_reference_voice_00001_.mp3 | PersonaPalette output generated from the source voice |

The sample connects LoadAudio to PersonaPalette, then saves both persona_ref_audio and the DSP preview as MP3 files. If the imported workflow cannot find the audio file in your ComfyUI input directory, load examples/F2_surprise_regular_33.wav manually in the LoadAudio node.

Sample Audio Attribution

examples/F2_surprise_regular_33.wav is from the JVNV corpus, "JVNV: A Corpus of Japanese Emotional Speech with Verbal Content and Nonverbal Expressions", by Detai Xin, Junfeng Jiang, Shinnosuke Takamichi, Yuki Saito, Akiko Aizawa, and Hiroshi Saruwatari. The JVNV corpus is licensed under CC BY-SA 4.0.

examples/new_reference_voice_00001_.mp3 is an adapted sample generated from F2_surprise_regular_33.wav with PersonaPalette. It is distributed under CC BY-SA 4.0 as an adaptation of the JVNV corpus sample.

Current Status

This project is in Phase 1. The API and persona mappings are still experimental.

The current nodes are:

PersonaPalette
Irodori-TTS Ref Connector
Unload PersonaPalette

Use Irodori-TTS Ref Connector when you want to connect persona_ref_audio to an IrodoriTTS Sampler or IrodoriTTS-v2 Sampler ref_audio_config input.

Use Unload PersonaPalette when you want to explicitly release cached PersonaPalette models after a run.

PersonaPalette

Inputs

| Input | Type | Default | Notes | | --- | --- | ---: | --- | | audio | AUDIO | required | Source voice audio | | youth | FLOAT | 0.0 | Youthful / settled direction | | brightness | FLOAT | 0.0 | Bright / dark direction | | energy | FLOAT | 0.0 | High-energy / low-energy direction | | softness | FLOAT | 0.0 | Soft / hard direction | | composure | FLOAT | 0.0 | Clear, slow, composed direction | | similarity_cfg | FLOAT | 0.5 | Seed-VC similarity CFG | | intelligibility_cfg | FLOAT | 0.5 | Seed-VC intelligibility CFG | | diffusion_steps | INT | 50 | Seed-VC diffusion steps |

Persona axes accept -1.0 to +1.0.

For normal use, start with persona axes in the -0.5 to +0.5 range. Values near -1.0 or +1.0 are strong exploration settings.

Outputs

| Output | Type | Notes | | --- | --- | --- | | persona_ref_audio | AUDIO | Seed-VC processed persona reference audio | | dsp_preview_audio | AUDIO | DSP-only intermediate audio before Seed-VC | | metadata_json | STRING | Parameters, DSP values, Seed-VC settings, and audio info |

dsp_preview_audio is useful for checking how much the persona axes are changing the source before Seed-VC naturalization.

Irodori-TTS Ref Connector

Irodori-TTS Ref Connector converts persona_ref_audio into reference config outputs for Irodori-TTS sampler nodes.

Related Irodori-TTS projects:

• Aratako/Irodori-TTS-500M: Irodori-TTS 500M model.
• Aratako/Irodori-TTS-500M-v2: Irodori-TTS 500M v2 model.
• jupo-ai/comfy_IrodoriTTS: ComfyUI wrapper for Irodori-TTS.
• kantan-kanto/ComfyUI_IrodoriTTS-v2_Wrapper: ComfyUI wrapper for Irodori-TTS v2.

Inputs:

| Input | Type | Default | Notes | | --- | --- | ---: | --- | | persona_ref_audio | AUDIO | required | Connect from PersonaPalette persona_ref_audio | | normalize_ref_audio | BOOL | False | When enabled, sets ref_normalize_db to -16.0 | | max_ref_seconds | FLOAT | 30.0 | Maximum reference length passed to Irodori-TTS |

Outputs:

| Output | Type | Notes | | --- | --- | --- | | irodori_ref_config | IRODORI_REF_CONFIG | For IrodoriTTS Sampler | | irodori_v2_ref_config | IRODORI_V2_REF_CONFIG | For IrodoriTTS-v2 Sampler |

Unload PersonaPalette

Unload PersonaPalette is an optional passthrough output node for freeing cached PersonaPalette model memory after a workflow run.

Set unload_now=true and queue the node to unload the cached Seed-VC models. Repeated PersonaPalette runs are faster when the models stay loaded, so use this node only when you need to free VRAM for other work.

Inputs:

| Input | Type | Default | Notes | | --- | --- | ---: | --- | | unload_now | BOOL | False | When true, unloads cached PersonaPalette models | | trigger | * | optional | Passthrough input used to control execution order |

Outputs:

| Output | Type | Notes | | --- | --- | --- | | trigger | * | Returns the passthrough input unchanged |

Recommended Starting Settings

similarity_cfg = 0.5
intelligibility_cfg = 0.5
diffusion_steps = 50
persona axes = -0.5 to +0.5

Use dsp_preview_audio and persona_ref_audio together:

• If DSP changes are too subtle, increase the persona axis value.
• If Seed-VC changes accent or prosody too much, lower similarity_cfg.
• If pronunciation becomes too hard or artificial, try lowering intelligibility_cfg.
• CPU execution can be slow, especially for longer audio.

Installation

Clone this repository into your ComfyUI custom_nodes directory:

cd ComfyUI/custom_nodes
git clone https://github.com/kantan-kanto/ComfyUI-PersonaPalette

Install Python dependencies in the same Python environment used by ComfyUI:

pip install -r ComfyUI-PersonaPalette/requirements.txt

Restart ComfyUI after installation.

Seed-VC Model Files

PersonaPalette does not auto-download Seed-VC models. Download the files manually and place them under:

ComfyUI/models/seedvc/

Required files:

| Place under | Download from | Required file(s) | | --- | --- | --- | | ComfyUI/models/seedvc/v2/ | Plachta/Seed-VC v2 | ar_base.pth, cfm_small.pth | | ComfyUI/models/seedvc/hf/whisper-small/ | openai/whisper-small | Entire repository contents | | ComfyUI/models/seedvc/hf/hubert-large-ll60k/ | facebook/hubert-large-ll60k | Entire repository contents | | ComfyUI/models/seedvc/hf/ASTRAL-quantization/ | Plachta/ASTRAL-quantization | bsq32/bsq32_light.pth, bsq2048/bsq2048_light.pth | | ComfyUI/models/seedvc/hf/campplus/ | funasr/campplus | campplus_cn_common.bin | | ComfyUI/models/seedvc/hf/bigvgan_v2_22khz_80band_256x/ | nvidia/bigvgan_v2_22khz_80band_256x | config.json, bigvgan_generator.pt |

If you use git clone for the Hugging Face repositories, keep the cloned folder names as shown above. PersonaPalette also accepts common git-clone style names such as whisper-small, hubert-large-ll60k, ASTRAL-quantization, campplus, and bigvgan_v2_22khz_80band_256x.

Notes

• Seed-VC output is commonly 22050 Hz, so generated WAV files may be smaller than source or DSP preview files recorded at 44100 Hz or 48000 Hz.
• persona_ref_audio is AUDIO output. Use Irodori-TTS Ref Connector if a downstream Irodori-TTS sampler needs a ref_audio_config.
• The persona axes are empirical audio-design controls, not strict psychological measurements.

Third-Party Code

This repository vendors and adapts parts of Seed-VC. See AUTHORS.md for third-party attribution and local modification notes.

Release Notes

See CHANGELOG.md for detailed version history.

Current Version: v0.1.1

This release improves ComfyUI Registry compatibility by replacing Seed-VC environment-variable runtime configuration with explicit local model path injection.

Highlights:

• Removed environment variable reads and writes flagged by Registry scanning.
• Preserved local Seed-VC model loading through ComfyUI/models/seedvc/.
• Preserved BigVGAN huggingface_hub compatibility and CUDA/FP16 CFM dtype handling.