SAM-Audio Visual Separate

This is the party trick node, and it works: feed it an audio track, a sequence of video frames, and a mask, and it isolates the sound of the thing you masked. Two people talking, you white out one of them - out comes just that voice. A street scene, you mark the visible instrument - out comes the instrument. SAM-Audio is one of the few models that can do "the sound of the thing I'm pointing at," and this node is the ComfyUI version of pointing.

How it works

You give it three things plus a description:

• audio - the mixture, from ComfyUI's audio loader.
• images - the video frames, in chronological order, as an IMAGE batch.
• mask - a MASK where white selects the object whose sound you want. A single mask is automatically repeated across every frame, so you don't need to draw a moving region by hand for a static shot. If the object moves, you can supply one mask per frame instead.

The mask isn't a crop - the node turns your white region into a visual prompt by blacking out the same pixels before the model encodes the frames, and the model learns to associate "what's in this shape" with "what's in this audio." The description input is optional here; if you know the sound ("piano", "crowd") it helps the model lock on, but the mask is doing the real work.

One model note: the checkpoint you load matters. The -tv variants (text-visual) are the ones built for this path - if your Visual Separator results come out confused and you're on a plain sam-audio-large, switch the loader to the matching -tv model. The shared inputs are the usual suspects: seed, inference_steps (32 default), and the chunk_duration / chunk_overlap pair for long clips.

Outputs are target and residual, exactly like the text and span separators - wire both to PreviewAudio or a save node. Everything comes out mono at 48 kHz.

The workflow

Load Image ─────────────► SAM-Audio Visual Separate
Load Audio ─────────────► SAM-Audio Visual Separate
SAM-Audio Model Loader ─► SAM-Audio Visual Separate
                          └─► PreviewAudio / SaveAudio (target)
                          └─► PreviewAudio / SaveAudio (residual)

The -tv loader default is the main new thing to remember versus the text-only nodes.

Common issues

• Mask covers the wrong thing. White is the target, black is everything else - and because the prompt is encoded by blacking out pixels, a sloppy mask that half-includes a second source drags it in. Tight masks win.
• Model confusion. Wrong checkpoint flavor (non--tv) is the first thing to check.
• Frames and audio out of sync. The frames are matched to the audio positionally; if you feed a frame sequence that doesn't correspond to the audio timeline, the separation is nonsense. Keep them aligned.
• First-run downloads. Same story as the whole pack: checkpoint plus the pinned T5 encoder, and the -tv models add the visual encoder assets. One slow queue, then it's local.

Install via ComfyUI Manager ("ComfyUI-SAM-Audio") or the clone + install.py route in the pack README. If your source is already a video file with an audio track embedded, you don't need this node's audio + images split - the Video Separate node takes a native VIDEO value and does all of that for you.