ComfyUI-SAM-Audio

ComfyUI SAM-Audio

ComfyUI nodes for Meta SAM-Audio, which separates a described sound from an audio mixture.

Supports text prompts, positive and negative time spans, and visual masks. Each separator outputs the isolated target and the remaining audio.

Requirements

• Python 3.11 or newer
• A CUDA GPU is strongly recommended
• An internet connection for the first model download

Installation

Clone the repository into ComfyUI/custom_nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/ethanfel/ComfyUI-SAM-Audio.git

Run the installer with the same Python environment used by ComfyUI:

python ComfyUI-SAM-Audio/install.py

Then restart ComfyUI. If installed through ComfyUI Manager, the installer should run automatically.

The node uses ComfyUI's decoded media tensors and does not install or load TorchCodec.

Model download

Select a model in SAM-Audio Model Loader and queue the workflow. The loader first tries Meta's official repository. If Hugging Face denies gated access, it automatically downloads the matching pinned and checksum-verified checkpoint from the public mirror collection.

You can authenticate in the ComfyUI environment if you prefer the official repository:

hf auth login

Models download on first use to the active ComfyUI models directory under models/sam_audio/. For example, when ComfyUI is started with --models-directory /path/to/models, SAM-Audio checkpoints are stored in /path/to/models/sam_audio/. Both official and mirrored checkpoints remain subject to Meta's SAM License.

The required T5 asset is pinned in the Hugging Face cache and reused locally after its first download. The separate 5.8 GiB PE span predictor is loaded only when predict_spans is enabled.

Nodes

• SAM-Audio Model Loader — loads an official or local checkpoint
• SAM-Audio Text Separate — separates audio using a text description
• SAM-Audio Span Prompt — creates positive or negative time spans
• SAM-Audio Span Separate — separates audio using text and time spans
• SAM-Audio Visual Separate — separates audio using image frames and a mask
• SAM-Audio Video Separate — accepts a native ComfyUI VIDEO, uses its embedded audio, and separates the object selected by a mask

All nodes are in audio/SAM-Audio.

Basic use

1. Load a model with SAM-Audio Model Loader.
2. Load audio with ComfyUI's audio loader.
3. Connect both to a separator and enter a short prompt such as man speaking or dog barking.
4. Preview or save the target and residual outputs.

Long audio is processed in overlapping chunks by default. Set chunk_duration to 0 for a single full-clip pass.

Example workflows are included in the examples directory.

Notes

• Output is mono at 48 kHz.
• Adjacent chunks use a normalized crossfade; the defaults are 10-second chunks with 1-second overlap.
• White pixels in a visual mask identify the target object.
• Native VIDEO inputs must contain an audio track. Use ComfyUI's Create Video node to attach separate audio when needed.
• Local model folders must contain config.json and checkpoint.pt.
• The model loader offers PyTorch SDPA (default) or Comfy Kitchen INT8 attention. Comfy Kitchen requires a supported current ComfyUI build and GPU and may slightly change results because its attention is quantized.
• ComfyUI's Unload Models action moves SAM-Audio out of VRAM but keeps the cached loader output in system RAM. Use Unload Models and Execution Cache when you also want ComfyUI to discard that cached CPU model.

License

This ComfyUI integration is released under the MIT License. SAM-Audio and its checkpoints use Meta's separate SAM License.