ComfyUI Node
Muse Voice Swap V1
Separates vocals from ambient in an LTX-generated audio track, re-voices the detected speech segment(s) via Fish S2 cloning, and remixes with the untouched ambient bed.
Muse Voice Swap V1
- source_audio
- reference_audio
- reference_audio_2
- reference_audio_3
- final_audio
- new_vocals_only
- bg_audio_only
- speaker_2_vocals_only
- speaker_3_vocals_only
- timeline_data
◄timeline_data{}►
◄use_timeline_overridefalse►
◄multi_speaker_modefalse►
◄auto_assign_speakersfalse►
◄sep_chunk_length10.0►
◄sep_chunk_overlap0.10►
◄sep_chunk_fade_shapelinear►
◄vad_top_db35►
◄min_segment_seconds0.30►
◄min_gap_seconds0.20►
◄whisper_model_sizesmall►
◄whisper_languageauto►
◄use_llm_emotion_tagsfalse►
◄gemma_clip_name(no text_encoders found)►
◄emotion_tag_temperature0.70►
◄fish_model_path▾►
◄fish_languageauto►
◄fish_deviceauto►
◄fish_precisionauto►
◄fish_attentionauto►
◄fish_max_new_tokens0►
◄fish_chunk_length200►
◄fish_temperature0.80►
◄fish_top_p0.80►
◄fish_repetition_penalty1.10►
◄fish_seed0►
◄fish_keep_model_loadedtrue►
◄time_match_moderipple►
◄stretch_fft_size2048►
◄bg_extend_modehold_last►
◄bg_volume1.00►
◄vad_methodsilero►
◄silero_threshold0.50►
◄silero_speech_pad_ms30►
◄timeline_ui►
◄min_confidence_margin0.10►
◄speaker_aware_mergefalse►
◄merge_similarity_threshold0.30►
◄reference_text►
◄reference_text_2►
◄reference_text_3►
CategoryMuse Collective
Inputs (45)
| Name | Type | Default | Description |
|---|---|---|---|
| source_audio | AUDIO | The LTX-generated mixed audio (speech + ambient). | |
| reference_audio | AUDIO | Voice-clone target reference, 5-30s. | |
| timeline_data | STRING | {} | — |
| use_timeline_override | BOOLEAN | false | ON = keep existing segment boundaries/text from timeline_data; skip auto-detect and auto-transcribe, only re-clone + remix. |
| multi_speaker_mode | BOOLEAN | false | ON = assign each segment to speaker 1/2/3 (set per-segment in the timeline panel) and clone it using that speaker's reference voice (reference_audio / reference_audio_2 / reference_audio_3). Also produces isolated audio tracks for speakers 2 and 3. |
| auto_assign_speakers | BOOLEAN | false | ON (with multi_speaker_mode) = auto-guess each segment's speaker by comparing its voice against reference_audio/_2/_3 using a speaker-embedding similarity match. This is a best-guess classifier, not a guarantee — review/correct assignments in the timeline afterward, same as you would Whisper's transcript. Only runs on a fresh (non-override) pass — once use_timeline_override is on, your manual corrections are trusted as-is and this is skipped entirely, so you don't need to lock a segment just to protect a manual speaker fix. |
| sep_chunk_length | FLOAT | 10.02–60 | — |
| sep_chunk_overlap | FLOAT | 0.100–2 | — |
| sep_chunk_fade_shape | COMBO | linear | 4 options: linear, half_sine, logarithmic, exponential |
| vad_top_db | FLOAT | 355–60 | — |
| min_segment_seconds | FLOAT | 0.300.05–5 | — |
| min_gap_seconds | FLOAT | 0.200.02–5 | — |
| whisper_model_size | COMBO | small | 5 options: tiny, base, small, medium, large-v3 |
| whisper_language | COMBO | auto | 12 options: auto, en, zh, ja, ko, es, +6 |
| use_llm_emotion_tags | BOOLEAN | false | ON = feed each segment's actual audio to a local Gemma model and ask it to insert Fish S2-style emotion/prosody tags ([excited], [whisper], [pause], etc.) based on the real delivery, before cloning. Fixes flat/ monotone clones. Fully local — no API key, no quota/cost. |
| gemma_clip_name | COMBO | (no text_encoders found) | Gemma checkpoint from ComfyUI/models/text_encoders/, loaded via the same CLIPLoader + TextGenerate nodes ComfyUI uses natively. e.g. gemma4_e4b_it_fp8_scaled.safetensors |
| emotion_tag_temperature | FLOAT | 0.700–2 | — |
| fish_model_path | COMBO | 1 options: (FishAudioS2 not loaded) | |
| fish_language | COMBO | auto | 44 options: auto, en, zh, ja, ko, es, +38 |
| fish_device | COMBO | auto | 4 options: auto, cuda, cpu, mps |
| fish_precision | COMBO | auto | 4 options: auto, bfloat16, float16, float32 |
| fish_attention | COMBO | auto | 4 options: auto, sdpa, sage_attention, flash_attention |
| fish_max_new_tokens | INT | 00–4096 | — |
| fish_chunk_length | INT | 200100–400 | — |
| fish_temperature | FLOAT | 0.800.1–1 | — |
| fish_top_p | FLOAT | 0.800.1–1 | — |
| fish_repetition_penalty | FLOAT | 1.100.9–2 | — |
| fish_seed | INT | 00–2147483647 | — |
| fish_keep_model_loaded | BOOLEAN | true | — |
| time_match_mode | COMBO | ripple | 3 options: ripple, stretch_to_fit, off |
| stretch_fft_size | INT | 2048512–8192 | — |
| bg_extend_mode | COMBO | hold_last | 3 options: hold_last, loop, trim |
| bg_volume | FLOAT | 1.000–2 | — |
| vad_method | COMBO | silero | silero (recommended) = neural voice-activity model, correctly tells quiet-but-real speech apart from actual silence. energy_threshold = the original librosa.effects.split method (relative dB threshold) — kept as an instant revert path if silero doesn't suit some source material. min_segment_seconds/min_gap_seconds above apply to either; vad_top_db only applies to energy_threshold. |
| silero_threshold | FLOAT | 0.500.1–0.9 | Only applies when vad_method is silero. Speech-probability cutoff Silero requires before calling a chunk of audio speech at all -- LOWER this if it's missing quiet/soft-spoken talking (more sensitive, but risks catching more non-speech noise too). Separate from min_segment_seconds/min_gap_seconds, which only filter/merge what Silero already decided was speech. |
| silero_speech_pad_ms | INT | 300–500 | Only applies when vad_method is silero. Padding (ms) added to both ends of each detected speech chunk -- raise this if word onsets/tails right at a segment boundary are getting clipped/missed. |
| timeline_ui | STRING | — | |
| min_confidence_margin | FLOAT | 0.100–0.9 | Only applies with multi_speaker_mode + auto_assign_speakers. How much the top speaker match must beat the runner-up (cosine similarity) before a segment's speaker gets auto-assigned. LOWER = more segments get auto-assigned (more risk of a wrong guess); HIGHER = fewer, more confident assignments (more segments left unchanged for manual review). 0.1 was previously hardcoded; short (<1s) segments tend to score low confidence regardless of this value — that's an ECAPA-TDNN limitation on short clips, not something this setting fixes. |
| speaker_aware_merge | BOOLEAN | false | ON = before merging two speech intervals separated by a gap shorter than min_gap_seconds, check whether they sound like the same voice first. Only merges when they do — a brief interjection from someone else (an 'ouch!', 'help!', a quick reaction) wedged into a small gap stays as its own segment instead of being silently absorbed into the surrounding speaker's segment, which is what happens with this OFF (the original behavior). Compares the two intervals directly against each other, not against any reference voice — works independently of multi_speaker_mode/auto_assign_speakers, and is useful even with a single known speaker (keeps an unrelated voice from merging into their segment). Falls back to always-merge whenever the check itself can't be computed. |
| merge_similarity_threshold | FLOAT | 0.300–0.9 | Only applies with speaker_aware_merge ON. How similar two adjacent intervals' voices must be (cosine similarity) to still merge across a short gap. LOWER = merges more readily (risks absorbing a real interjection); HIGHER = splits more readily (risks splitting one person's own natural pause into two segments). |
| reference_audio_2opt | AUDIO | Reference voice for segments assigned speaker 2 (multi_speaker_mode). | |
| reference_audio_3opt | AUDIO | Reference voice for segments assigned speaker 3 (multi_speaker_mode). | |
| reference_textopt | STRING | Transcript of reference_audio. Leave disconnected to auto-transcribe it internally via Whisper — no need to wire anything in for the normal case. | |
| reference_text_2opt | STRING | — | |
| reference_text_3opt | STRING | — |
Outputs (6)
| Name | Type | Description |
|---|---|---|
| final_audio | AUDIO | — |
| new_vocals_only | AUDIO | — |
| bg_audio_only | AUDIO | — |
| speaker_2_vocals_only | AUDIO | — |
| speaker_3_vocals_only | AUDIO | — |
| timeline_data | STRING | — |