Nodes/comfyui-fvmtools/Sign Selector SAM3
ComfyUI Node

Sign Selector SAM3

SAM3 text grounding for the sign you can't read

By ping1979ping·Created 6 months ago·Updated about a month ago· 0
Sign Selector SAM3
  • sam3_model
  • image
  • restrict_mask
  • sign_data
  • masks
  • crops
  • preview
  • region_count
  • report
class_signtrue
class_labeltrue
class_garment_printtrue
class_postertrue
class_screentrue
class_booktrue
class_platetrue
class_papertrue
class_graffititrue
custom_prompts
threshold_scale1.00
min_height_px24
min_area_ratio0.0005
max_regions12
merge_iou0.50
slop_detectionocr
slop_threshold0.50
only_slopfalse
cluster_similartrue
cluster_distance0.15
sort_orderarea_desc
ocr_backendauto

Generated images are full of text that isn't text. Street signs that read like someone sneezed on the keyboard, bottles with lettering that almost spells something, shopfronts full of confident nonsense. Sign Selector SAM3 is the node that finds that nonsense - and finding it is the half of the problem everyone skips. You can't inpaint your way to legible lettering if you can't tell the pipeline where the lettering is.

It's the first stage of FVMtools' Sign Tools, a four-node text-repair pipeline. It grounds nine built-in text classes - sign, label, garment print, poster, screen, book, license plate, paper, graffiti - against the image using SAM3 text grounding, gates them by minimum readable height, and hands the surviving regions downstream as SIGN_DATA. Think of it as a detailer's detector, but for text-bearing surfaces instead of faces.

Why enabling all nine classes is nearly free

The clever bit is that every enabled class shares ONE SAM3 vision encode per image. The vision backbone is the dominant cost of a SAM3 call, and the code runs it once, then grounds all nine prompts against the same cached image embedding. Enabling everything costs about the same as enabling one - and you can add your own via custom_prompts in the same prompt:threshold format (neon sign:0.3), which is also how you nudge SAM3 to mask the whole surface rather than just the lettering row.

Slop detection: the part nobody else does

The selector doesn't just find text, it judges it. With an optional OCR backend installed, each region gets a slop score - how implausible the existing lettering is - and the strongest signal is a contradiction: SAM3 says "there is text here" while OCR reads nothing. That's the classic pseudo-glyph signature, far more reliable than eyeballing. OCR is optional; without it the judgement falls back to the vision model, and nothing errors - you just get a hint in the report.

Inputs you'll actually touch

  • sam3_model - from the LoadSAM3Model node, which ships with a separate SAM3 pack (the comfyui-sam3 folder), not with FVMtools. You also need the gated sam3.pt checkpoint in ComfyUI/models/sam3/.
  • threshold_scale - the first knob to reach for. Below 1.0 finds more (and more false positives), above 1.0 is stricter.
  • only_slop - on, drop regions whose text already looks fine. Off, keep everything and let the Detailer decide. If you're only here to fix broken lettering, this pays for itself.
  • cluster_similar - a shelf of twelve identical bottles becomes one decision instead of twelve, which the whole pipeline then treats as one.

Outputs

sign_data feeds Sign Text Proposer. masks, crops and a preview let you see what it found (the preview numbers are what manual overrides later refer to), and region_count plus a report tell you what happened. There's also an optional restrict_mask if you only want to scan part of the frame.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/ping1979ping/comfyui-FVMtools
git clone https://github.com/wouterverweirder/comfyui_sam3   # for LoadSAM3Model

Then grab the SAM3 checkpoint (it's gated on HuggingFace, so hf auth login first) and, optionally, the OCR backend for slop detection:

python scripts/fetch_ocr_models.py

Restart ComfyUI after both clones. One honest warning: SAM3 is a heavy add and its ComfyUI nodes are known to need Triton, which is a real obstacle on Windows.

Troubleshooting

No regions found: lower threshold_scale toward 0.7 and drop min_height_px. Too many false positives: raise the scale, disable classes you don't need, cut max_regions. Found everything but nothing worth fixing: that's exactly what only_slop is for.

CategoryFVM Tools/Text

Inputs (25)

NameTypeDefaultDescription
sam3_modelSAM3_MODEL_CONFIGSAM3 model from the LoadSAM3Model node
imageIMAGEImage(s) to scan for text regions. Batch supported.
class_signBOOLEANtrueGround 'sign' — prompts: sign, street sign, shop sign (default threshold 0.3, min height 32px)
class_labelBOOLEANtrueGround 'label' — prompts: bottle label, product label, packaging label (default threshold 0.28, min height 24px)
class_garment_printBOOLEANtrueGround 'garment_print' — prompts: printed text on clothing, t-shirt print, logo on shirt (default threshold 0.3, min height 40px)
class_posterBOOLEANtrueGround 'poster' — prompts: poster, banner, billboard (default threshold 0.28, min height 40px)
class_screenBOOLEANtrueGround 'screen' — prompts: phone screen, computer monitor, display screen (default threshold 0.3, min height 32px)
class_bookBOOLEANtrueGround 'book' — prompts: book cover, magazine cover (default threshold 0.3, min height 32px)
class_plateBOOLEANtrueGround 'plate' — prompts: license plate (default threshold 0.35, min height 20px)
class_paperBOOLEANtrueGround 'paper' — prompts: document, menu, price tag, receipt (default threshold 0.28, min height 24px)
class_graffitiBOOLEANtrueGround 'graffiti' — prompts: graffiti, handwritten text (default threshold 0.3, min height 40px)
custom_promptsSTRINGExtra SAM3 prompts beyond the built-in classes. Format: 'neon sign:0.25, bottle label:0.3' — the threshold is optional.
threshold_scaleFLOAT1.000.3–2Multiplies every class's default threshold. Below 1.0 finds more (and more false positives), above 1.0 is stricter.
min_height_pxINT244–512Global floor for text height in the ORIGINAL image (min-area-rect short side). Per-class minimums still apply on top of this.
min_area_ratioFLOAT0.00050–0.5Region must cover at least this fraction of the image area.
max_regionsINT121–100Cost brake — keeps only the top N regions after sorting.
merge_iouFLOAT0.500–1Two classes hitting the same object are merged above this IoU. The higher-scoring detection keeps its class.
slop_detectionCOMBOocrHow to judge whether existing lettering is believable. - ocr: OCR confidence + dictionary + bigram plausibility - vlm: leave the judgement to the Proposer's vision model - ocr+vlm: both, combined in the Proposer Falls back gracefully when no OCR backend is installed.
slop_thresholdFLOAT0.500–1Regions scoring at or above this are marked as needing a re-render.
only_slopBOOLEANfalseON: drop regions whose text already looks fine. OFF: keep everything and let the Detailer decide.
cluster_similarBOOLEANtrueGroup near-identical regions (a shelf of identical bottles) so they share one text decision.
cluster_distanceFLOAT0.150.01–0.9Lower = stricter grouping. Combines perceptual hash and colour signature.
sort_orderCOMBOarea_descOrder of regions — also the order in which the Detailer renders them.
restrict_maskoptMASKOnly search inside this mask. Regions must overlap it by 30%+.
ocr_backendoptCOMBOautoOCR engine for slop detection. 'auto' picks the first installed one. Missing models degrade to VLM-only judgement, never an error.

Outputs (6)

NameTypeDescription
sign_dataSIGN_DATA
masksMASK
cropsIMAGE
previewIMAGE
region_countINT
reportSTRING