JB · MOSS Layout from SAM3

What this is for

MOSS-Image doesn't read tag soup. It reads a JSON layout: a summary (scene caption, meta tags, canvas kind) plus items, each with a label - character, object or text - a caption and a box. Great if you want two specific people in specific places, miserable to hand-write, because those boxes are pixels of the render canvas, measured from its centre, ordered yxyx. Nobody eyeballs that.

So this node does the eyeballing. You hand it a photo whose composition you like and name what to look for in it; it finds the boxes, fills each from your prompt, writes the JSON. The photo donates arrangement and aspect ratio only - the render is a fresh text-to-image, no style bleed, no img2img.

This is the layout branch of composition control, competing with a ControlNet reference pass. A ControlNet drags the reference's edges into your latent whether you wanted the lamp or not; a layout says "a character box here, a table box there, the rest is this" and lets the model fill the gaps. The catch: you have to be able to name what you're placing.

That naming is SAM3 doing promptable concept segmentation; this pack keeps only the boxes.

How it works, mechanically

SAM3 encodes the reference once - on the native path the trunk runs a single time and every rule grounds against that cached state, so ten rules cost one encode, not ten.

Each line of box_rules becomes a phrase, e.g. person. Hits come back with scores and masks; the masks are dropped. Per rule the node does best-first NMS at IoU 0.6 (no three boxes stacked on one person), drops anything under min_area, keeps the top max_per_rule, sorted left to right by box centre. {n} in the slot becomes 1, 2, 3 in that order.

Each box's slot names a key in your prompt JSON, and that sub-tree becomes the box's caption, flattened into sentences. What the boxes don't claim - usually the location - becomes the summary caption, after high_level_description. Boxes are then converted to MOSS's abs_centralized yxyx: scale to canvas, floor top/left, ceil bottom/right, subtract the canvas centre.

Then it all runs twice, because MOSS boxes are pixels of the canvas they're rendered on: moss_prompt for width/height, moss_prompt_upscaled re-emitting the same boxes at up_width/up_height.

The inputs that matter

box_rules is the one you'll edit - one rule per line, pipe-separated, # starts a comment:

person | character | character_{n}
table  | object    | table

Label falls back to object if blank or mistyped, and the slot is optional: no slot, and the box is captioned with its raw SAM3 prompt. prompt is a force-input - wire a Stitcher's raw_json into it. A single wrapper key (the Stitcher's title, like scene) gets unwrapped before slots are matched.

The rest is tuning: threshold (0.4) gates every rule; max_per_rule (3) and min_area (0.01) keep crowds and specks out; megapixels (1.0) sizes pass 1 at the reference's aspect, 16 px snapped, and 0 means "reference size", rarely what you want; upscale_by (1.5) sets the refine canvas; meta is your medium tags; canvas is single/multi_panel/collage.

Outputs: moss_prompt into pass 1's CLIPTextEncode, moss_prompt_upscaled into the refine pass, width/height and up_width/up_height into your latent and scale nodes, preview (the reference at canvas size with every box, slot and score drawn on it), and report.

Install

Manager → search comfyui-fvmtools → install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/ping1979ping/comfyui-FVMtools

Don't copy the README's clone line verbatim - it still says your-username/comfyui-FVMtools. Its requirements.txt pulls insightface, onnxruntime-gpu and opencv for the Face Tools; this node needs none of it, just the numpy and Pillow ComfyUI ships.

What you do need is SAM3. Easy way: put sam3.1_multiplex_fp16.safetensors in models/checkpoints, load it with a CheckpointLoaderSimple, and wire its MODEL to sam3_native_model and its CLIP to sam3_native_clip from that same loader - the text encoder lives inside the checkpoint. Otherwise the comfyui-sam3 extension's Load SAM3 Model goes into sam3_model; native wins if both are wired.

MOSS-Image itself, its VAE and the turbo LoRA aren't in this pack.

Where people get burned

Errors don't raise - they land in report. With no SAM3 wired you get ERROR: no SAM3 wired … plus an items: [] layout, which renders happily as a plain text-to-image picture with no layout in it. Read report before blaming the checkpoint.

Slot names must match exactly. A box captioned with the bare word person found no key by that name in your prompt - title Stitchers character_1, character_2 to suit. Numbering is strictly left to right, so if the left-hand person should be character_2, swap the wiring, not the rules.

text boxes place a box; they don't read. Nothing OCRs the sign, so the item's text field is empty and the caption has to come from the slot.

Wire the second prompt - moss_prompt_upscaled to the refine pass, up_width/up_height to the latent - or every box is off by your upscale factor.

Turbo discipline, from the author's own example graph: CFG 1, euler/simple, 4 steps at ≤1024², 6 at ~1536, flow shift 5. MOSS at a normal 25–50 steps turns to mush.