SCAIL-2 Reference SAM Builder

SCAIL-2 doesn't just take a reference image and call it a day - in replacement mode and multi-character setups it wants masks that tell it which pixel belongs to which subject. Building those by hand is exactly the kind of fiddly, easy-to-get-wrong plumbing this pack exists to hide. SCAIL2ReferenceSAMBuilder is the node that builds them for you: it takes a Reference Pack, runs SAM3 tracking over the reference images, and returns the pack enriched with the colored masks SCAIL-2 needs.

Honest positioning: this is the least-used node in the pack, and that's fine. The single-image path doesn't need it - SCAIL2SimpleVideo generates masks itself from SAM3_VideoTrack data when you give it track inputs. You reach for the SAM Builder when you're on the multi-reference workflow (2. SCAIL2_multi_ref.json), where the README explicitly recommends it. If you're only ever doing one reference character, you can skip it.

How it works

It chains the pieces you'd otherwise have to wire yourself. For each subject in the pack, it takes the subject's main image and runs SAM3_VideoTrack on it - the same core node the driving side uses - with your SAM model and conditioning prompt. Detections get filtered by detection_threshold, capped at max_objects, and re-detected every detect_interval frames. Each tracked subject is then turned into a colored reference mask (color-coded so SCAIL-2 can tell subjects apart) and packed back into the reference-pack object. The result plugs straight into the generation node.

The conditioning input is worth understanding because it's the one that trips people up: it's a CONDITIONING, not a prompt string. The source raises a clear error if you feed it something without a SAM3-usable condition, and the intended wiring is a CLIPTextEncode of a single class word like "person".

The inputs that matter

• reference_pack - the SCAIL2_REFERENCE_PACK from SCAIL-2 Reference Pack. Required.
• sam_model - a MODEL. Load the SAM3 checkpoint (the pack's example layout uses sam3.1_multiplex_fp16.safetensors) with UnetLoader or similar.
• conditioning - CLIPTextEncode of "person" (or the object class you're tracking).
• detection_threshold - default 0.5. Lower it if SAM3 is missing subjects, raise it if it's picking up junk.
• max_objects - default 2, up to 8. How many detections per frame to keep.
• detect_interval - default 1. Re-detect every N frames. For a static reference image this barely matters; for anything with motion it's your tracking cadence.

Outputs

• reference_pack - the enriched pack, same SCAIL2_REFERENCE_PACK type. Wire it into SCAIL2SimpleVideo.reference_image.
• summary - a STRING describing what got tracked and masked. Good for confirming each subject was actually found before you burn a generation.

Installing and gotchas

Part of ComfyUI-SCAIL2-Easy, so: ComfyUI Manager → search "ComfyUI-SCAIL2-Easy", or git clone https://github.com/jieg9341-lab/ComfyUI-SCAIL2-Easy into custom_nodes, restart. The node itself depends only on the pack's light requirements, but it calls SAM3_VideoTrack from comfy_extras, so you need a ComfyUI build where that's available (recent core or the SAM3 pack).

The gotcha to keep in your head is symmetric with the driving side: the README warns that for multi-subject or replacement mode, the driving video's SAM3_VideoTrack.max_objects has to be big enough to cover every person you want handled - this node covers the reference images, but the generation node separately needs driving track data with enough tracked objects. Set both sides' max_objects high enough and start with few references. That's the whole failure mode.