Florence2 (RMBG)

Type "the red dress" and get a mask of the red dress. That's the pitch, and Florence-2 mostly delivers on it. This node wraps Microsoft's Florence-2 vision-language model so you can point at objects with words instead of clicks or bounding boxes - which is the whole appeal when you're building an automated pipeline that has to find and mask specific things it hasn't seen yet.

It sits in the "targeted masking" corner of the background-removal world. A plain background remover answers "subject vs. background." Florence-2 answers "where is the thing I named," which is a different and harder question. If all you want is a clean cutout, this is overkill - reach for BiRefNet or the RMBG node. Florence-2 earns its keep when you need per-object selection to feed inpainting, regional prompting, or a compositing step.

How it works

Florence-2 is a small, genuinely capable vision model that does captioning, detection, and grounding from one set of weights. This node exposes the grounding side: you give it an image and a text prompt, it localizes what you asked for and returns a mask. It's the same family of tool as GroundingDINO (which the pack also bundles), just with a different model behind it - Florence-2 tends to be lighter and faster to load.

The inputs that matter

• text_prompt (optional, multiline) - what you're looking for. Tag-style ("cat, dog") or natural language both work. This is empty by default, so if you get nothing back, this is the first place to look.
• task - pick your output shape: Polygon Mask (text prompt) gives you a mask (the default, and what you want most of the time), Phrase Grounding (text boxes) gives boxes tied to phrases, Region Proposals (boxes only) just proposes regions with no text steering.
• model_name - six choices, from microsoft/Florence-2-base up to microsoft/Florence-2-large plus two community CogFlorence fine-tunes. Base is the fast default; large is more accurate and slower. The -ft variants are fine-tuned checkpoints. Start with base, move up only if base misses things.
• fill_mask (default on) - fills the detected polygon into a solid mask rather than leaving an outline. Usually what you want.

Outputs are IMAGE (annotated preview), MASK (the thing you actually wire into inpainting or a mask combiner), and DATA - a JSON blob of the raw detections. That DATA output is the interesting one: feed it into the pack's Florence2 Box Coordinates node to pull out bounding boxes and center points for cropping or region logic.

precision (fp16/bf16/fp32) and attention (flash_attention_2/sdpa/eager) are there for tuning - fp16 and sdpa are sane defaults; only touch them if you hit an error or want to squeeze speed. keep_model_loaded keeps the weights resident between runs, which saves reload time if you're iterating and have the VRAM to spare.

How to install it

• ComfyUI Manager: search Comfyui-RMBG, install, restart.
• Manual: cd ComfyUI/custom_nodes && git clone https://github.com/1038lab/ComfyUI-RMBG, then ./ComfyUI/python_embeded/python -m pip install -r requirements.txt, and restart.

The Florence-2 weights download automatically from HuggingFace the first time you run the node - expect a wait on that first execution while it pulls the model you picked. It needs a working transformers install, which the pack's requirements handle.

Common issues

The one that bites people across this whole pack is a 401 error on first model download. It's not a permissions problem with the repo - it's a stale HuggingFace token on your machine. Delete %USERPROFILE%\.cache\huggingface\token (and unset any HF_TOKEN / HUGGINGFACE_TOKEN env vars), then re-run; these are public models and download anonymously. This pack is under active development - SAM3, Florence-2 and YOLOv8 all landed across late 2025 into the v3.0.0 release - so if something's flaky, check you're on a current version before assuming the node is broken. And if flash_attention_2 throws, switch attention to sdpa; not every GPU/build has flash-attn available.