Refine Mask with Prior and Points

There's always one. Frame 47: subject turns, the tracker grabs the wrong shoulder, and your beautiful mask sequence has a hole in it. Retracking the whole shot to fix one frame is absurd, and hand-painting a mask in an image editor means fighting the resolution, the binarization, and the file format.

Refine Mask with Prior and Points is the surgical option. You feed it one frame, some click coordinates, and optionally the mask you'd otherwise have had, and it runs two SAM refinement passes and gives you back a single corrected mask.

What it does

It loads the SAM model onto the GPU, upscales your frame to SAM's 1008×1008 working resolution with bilinear sampling, and converts your coordinates from source pixel space to that 1008 space for you. Positive points are "this is the subject," negative points are "this isn't." Then it does two identical refinement passes through the model's segmenter, feeding each pass's logits into the next - which is what makes point-prompted segmentation converge instead of wobbling.

The prior_file is the interesting input: if you point it at a saved .npz mask sequence and give it a frame, it pulls that frame's mask, resizes it, and turns it into a soft logit prior (±6) before the refinement passes. That's mask-guided correction - you're telling SAM "roughly this, now sharpen it with my clicks" rather than starting from nothing. Without a prior you have to give it at least one positive point, and it says so.

Output is a single MASK at your source resolution, thresholded at zero, ready to be spliced into the sequence.

Inputs and outputs

Required, all of them:

• model - the SAM model, as a MODEL.
• image - exactly one frame. The node validates len(image) == 1 and refuses a batch, which is blunt but keeps the semantics honest.
• positive_coords / negative_coords - JSON strings like [{"x": 312, "y": 208}], in source pixel coordinates. This is the fiddly part: there's no interactive canvas in this pack, so you read coordinates off a preview or from the SAM detect node's output. Default [].
• prior_file - a filename relative to ComfyUI's input directory, or empty for no prior. Path traversal is rejected.
• frame - which frame of the prior archive to use as the prior. Note that this is decoupled from how you cropped image: if you sliced frame 47 out and pass frame: 0, you'll get a prior from the wrong frame. Wire the real index.

One output: MASK.

Install

Manager → H3 Relay → restart, or clone:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/akatz-ai/h3-relay.git

This one reaches deep into the SAM model internals (model.model.diffusion_model.forward_segment) - it isn't a wrapper around a documented node. That means it's tied to the SAM 3 implementation shipped in your ComfyUI version, and a big refactor of the native SAM nodes upstream could break it before anything else in H3 Relay breaks. If you're pinning ComfyUI for stability, note this node in your reasons.

ComfyUI 0.37.0+, a SAM 3.1 checkpoint, plus numpy for reading the prior archive.

Where people get burned

An include point or derived prior-mask anchor is required fires whenever positive_coords is empty - even if you supplied a prior. The check happens before the prior is loaded, so a prior alone won't satisfy it. Give it one point inside the subject; think of the prior as a hint about shape, not a substitute for telling it what to segment.

Prior mask frame is outside the saved timeline means your frame index doesn't exist in that archive. And Prior mask must be a staged input is the path guard: the file has to live under ComfyUI's input folder, so copy it there first.

Practical advice: don't use this node in the middle of a full-length render chain. Do the mask pass once with the cheap tools, save the sequence, find the bad frames from the report, fix exactly those with this node, re-save. It's a repair tool, not a per-frame pass - running it over 120 frames would be slow, and it'd be doing SAM's job badly.