VoxLock Isolate Voice

Everything else in ComfyUI's audio corner generates sound: a TTS model invents a voice, ACE-Step writes a song, MMAudio fakes foley. This node does the opposite. You hand it a recording you already have - an interview, a phone call, a podcast where both mics bled - and it keeps one voice and removes the other people talking over it. AUDIO in, AUDIO out, plus a JSON report.

That's a job nothing else in the stack does. If your lip-sync clip has a second person on the track, no amount of sampling fixes it: the avatar will happily animate the wrong mouth. Isolate first, then drive the face.

How it works

VoxLock is a small target-speaker isolation model by Arbaz Siddiqui, and this pack is an unofficial integration of the official voxlock package - the node calls it and adds nothing to the maths. It runs on ONNX Runtime, on the CPU, at roughly 12× real time in the author's measurements.

Your audio arrives as a [batch, channels, samples] waveform at whatever rate Load Audio read it; the node means the channels to make it mono and resamples to 16 kHz with soxr - exactly what voxlock.load_audio does to a file, which is why the output matches the command line bit for bit rather than approximately. Then VoxLock.isolate keeps the voice it was pointed at and suppresses the rest, stacked into one mono track per batch item.

The model gets its target one of two ways. Given a voice_sample, it locks onto that voice. Without one, it takes the loudest 3 seconds in the first 15 seconds of the input and keeps that speaker - the rule the official CLI uses. Which is fine right up until it isn't.

The inputs and outputs

Required: model from VoxLock Loader, audio from Load Audio, and output_sample_rate, a two-option combo - 16000 (default, the model's output as-is) or input (resampled back to your input's rate). Optional: voice_sample, another AUDIO.

Two things to know about that optional input. About 3 seconds is the sweet spot; a whole minute of the person doesn't help. And if your audio is a batch, the sample must be one item for all of them or one per item - anything else raises voice_sample has N items; give one, or one per audio item (M). Use it whenever someone else is louder in the opening 15 seconds, which on real recordings is most of the time.

output_sample_rate: input is not a quality control. It resamples a 16 kHz-band-limited signal back up so it lines up with the rest of your project; it does not put back frequencies the model never produced. If the result sounds dull next to your source, that's the model's bandwidth, not a setting you got wrong.

The outputs are audio - mono, 16 kHz (or your input rate), in phase with the input and at about its level, so you can lay it back under music or ambience - and report, a JSON string with output_sample_rate, a seconds total, and per-item input_seconds, chunk-run count, and whether the voice was automatic or given. Wire it into PreviewAny, as the shipped workflow does; it's the fastest way to confirm the node kept the voice you expected instead of the loud one.

Install

Manager, search VoxLock. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-VoxLock
python -m pip install -r ComfyUI-VoxLock/requirements.txt
python ComfyUI-VoxLock/tools/download_model.py --models-dir ../models/voxlock

The pip line installs voxlock==1.1.1 (which brings onnxruntime, soundfile, soxr). The download script pulls ~275 MB of ONNX from the Hugging Face repo at the revision 1.1.1 itself uses, SHA-256 checked; --precisions int8 gets you 79 MB if you only want the default. Nothing downloads at run time - the pack never calls the package's downloader - so a missing model folder is an error, not a wait. Restart, then open workflows/gui/isolate_with_sample.json.

Where people get burned

The wrong voice got kept. The loudest-3-seconds rule is the whole failure mode. Someone laughs, a chair scrapes at 0:04, someone says "sorry, go ahead" - that's your new target speaker. Add a voice_sample.

Nodes missing after install. This pack needs ComfyUI's V3 node API (v0.38.0 was used) and ships no NODE_CLASS_MAPPINGS fallback. On an older build it logs [VoxLock] not loaded: … and registers nothing. Update ComfyUI.

The FileNotFoundError about models/voxlock/onnx/. You downloaded one precision and selected another, or you pointed --models-dir at models/ instead of models/voxlock. Re-run the downloader; it verifies hashes and skips what's already good.

Comparing against the upstream demo clips. The isolated demos in the VoxLock repository are the same signal with inverted polarity as what 1.1.1 produces (they were made with an earlier version). If you null-test and get surprise, check phase before you file anything.

Music and singing. Not what it's trained for, and the README says so. Speech, one voice, and it stops knowing what to do outside that.

Interrupting a long file takes effect between chunk runs, one per 0.8 s of audio - quick, but not instant on a two-hour recording.