Recover Empty SAM Frames

Video tracking drops out. Not often, and not usually for long - but when the subject turns away, or something crosses in front of them, or the shot has a cut you didn't notice, SAM comes back with an empty mask for a handful of frames and your character-swap guide silently loses a person.

Recover Empty SAM Frames is a targeted repair. It does not retrack the clip and it does not touch a single frame that already has a mask. It finds the empty frames and runs fresh per-frame detection on exactly those.

How it works

It scans the first source_frames masks and collects every frame whose mask is empty - area zero, not "small". Then, for each one, it calls ComfyUI's native SAM 3 detect on the corresponding single image at a threshold of 0.35 with two refinement iterations, requesting individual masks, and applies the same selection rule you used originally, via the H3_SUBJECT_SELECTION record that Select Mask by Area or Position produces. That's why that opaque little output exists: without it, recovery would be guessing which of the three people in the frame is yours.

Then it reports. The report JSON lists empty_tracking_frames and recovered_frames - both the frames it found and the frames it believed it fixed. Wire it to a text preview if you're the kind of person who reads instead of assumes. (You should be.)

recover_empty_frames can be set to false, which turns the node into a pure inspector: it finds and reports empty frames without changing anything. That's the cheapest way to decide whether a mask sequence is worth saving.

There's also an interrupt check inside the per-frame loop, so a Cancel actually lands between frames instead of after the whole recovery pass.

Inputs and outputs

Required: masks (your tracked sequence), images (the prepared frames the tracker saw - resolution and length must agree), model (the SAM model, as a MODEL), conditioning (the CLIP text encoding of your subject description - the same one you gave detection originally; a different phrase here is a different subject), selection (wire this from Select Mask), source_frames (unpadded count), and recover_empty_frames (default true).

Two outputs: recovered_masks (MASK, same length and resolution as the input) and report (STRING). The mask goes on to expansion; the report goes wherever you'll see it.

Install

Manager → H3 Relay → restart, or:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/akatz-ai/h3-relay.git

You need ComfyUI's native SAM 3 nodes (this imports comfy_extras.nodes_sam3) and a SAM 3 checkpoint - the pack's graphs expect checkpoints/sam3.1_multiplex_fp16.safetensors. SAM 3 wants Triton for the fast path, and that's a known nuisance on Windows; if the native SAM nodes don't work on your box, this node can't help you. ComfyUI 0.37.0+.

The H3 weights (and their US/EU/UK/Korea territory exclusion) only matter further down the graph.

Where people get burned

Tracking must cover the complete source timeline means your mask batch is shorter than source_frames - you're passing the padded length or the wrong count. This node recovers frames; it can't invent them.

The one to genuinely watch for: recovery is per-frame detection, so it has no temporal memory. On a frame where your subject is half-occluded, SAM 3 may find a different person that matches your text prompt better, and the selection rule will happily pick it. That's why the honest workflow is recover → look at the expanded overlay → and if a recovered frame looks wrong, go fix that one frame by hand with Refine Mask with Prior and Points. Recovery is a bulk fix for a boring failure, not a quality upgrade.

And if the report says it found no empty frames, don't run it again expecting a different answer. Empty is empty - your problem is elsewhere, probably an expansion that's too small or a prompt that describes a different subject than you think.