ClearVoice Speech Separation

This is the node you came for. Feed it a recording with two people talking - a podcast, an interview, that meeting you're legally not supposed to be processing - and it returns two separate audio streams, one voice per output. The rest of the pack is just a model loader that hands this node its weights.

It's a speech separation model, not a generic "music source splitter" like Demucs. It's trained for two speakers at 16 kHz, on the MossFormer2 architecture from Alibaba's speech lab (same model family as ModelScope's ClearerVoice-Studio). Given a single mixed track it produces two clean-ish stems, which is exactly what you want for transcribing a conversation speaker-by-speaker or isolating one person's lines for a video edit.

How it works

Under the hood the node does a fair amount of grunt work before the model ever sees the audio:

1. It unpacks ComfyUI's standard AUDIO dict (a waveform tensor plus a sample_rate).
2. Stereo input gets collapsed to mono - the channels are averaged together. Worth repeating: no stereo image survives this, it's a mono pipeline by design.
3. If your sample rate doesn't match, it resamples with soxr to the rate you picked.
4. The audio goes to a temp WAV file, the model runs on it, and the two outputs get repacked into fresh AUDIO dicts and handed back.

Long recordings are decoded in 30-second segments (that's what one_time_decode_length and decode_window are set to in the config), so a whole hour of audio takes a while - especially on CPU. If you have a GPU it'll use it, and it picks whichever card has the most free VRAM.

The inputs that matter

There are three required inputs and you'll actually touch two of them:

• model - the CLEARVOICE_MODEL from Load ClearVoice Model. Wire the loader's model output straight in. The model is cached between runs, so you don't pay the first-load cost (and its auto-download) every time.
• audio - any node that outputs ComfyUI's AUDIO type. The usual way is VideoHelperSuite's audio loader, but anything emitting the standard {waveform, sample_rate} dict works.
• target_sample_rate - an enum of 8000 or 16000, default 16000. The model was trained at 16 kHz, so 8000 is just a downsampled output you'd only pick to save space. Leave it on 16000.

The outputs

Two AUDIO outputs:

• separated_audio_1 and separated_audio_2 - one voice each, mono, at your target sample rate. There's no semantic "this is speaker A" guarantee between runs; they're just the two separated stems. Wire them into whatever saves or plays ComfyUI audio - VHS's VideoCombine for muxing into a video, or an audio save/playback node from an audio pack.

Install and gotchas

cd ComfyUI/custom_nodes
git clone https://github.com/GuardSkill/ComfyUI-VoiceSep

or search "ComfyUI-VoiceSep" in ComfyUI Manager, then restart. Dependencies are heavy but mostly already present: torchaudio, soundfile, librosa, scipy, numpy, einops, rotary_embedding_torch, opencv-python, scenedetect, pydub and friends.

Where people get burned:

• Missing soxr. The node imports it at module load, but the pack forgot to put it in requirements.txt. If the nodes don't show up after install, pip install soxr (system libsoxr needed on some setups) is the fix.
• First run downloads the model. The checkpoint comes from Hugging Face (alibabasglab/MossFormer2_SS_16K) automatically on first use. No internet, no model.
• It's mono and it's two-speaker only. Mix of three people, or a stereo podcast you wanted to keep stereo? Wrong tool.
• The README is basically empty. The pack is a quiet, low-traffic wrapper - great when it works, but you're on your own for support, and the code has rough edges (unused debug prints, a config pointing at paths that don't exist yet). If separation quality matters to you, verify with a clean two-person clip before trusting it in a real pipeline.