Florence2 Model Loader 🐑

This node loads Florence-2, Microsoft's little vision-language model, and hands it to the describe/caption nodes in this pack. Florence-2 is one of the two captioners the community actually reaches for (JoyCaption being the other), and it's the one I'd point a beginner at first - it's tiny, fast, and it doesn't need a monster GPU. The loader does one job: pick a variant, set precision, and output a model.

Why Florence-2 is the easy pick

Florence-2 is small - the base is ~0.23B parameters, the large is ~0.77B. That's nothing next to a 7B captioner, which means it loads in seconds and runs comfortably on modest cards. It's a solid general captioner on its own and, per the training community, it pairs especially well with a WD14 tagger when you want both a natural-language sentence and comma tags. It's also the standard advice for "stop using BLIP" - BLIP's captions are formulaic and inaccurate, and have been the wrong choice for years. Florence-2 or JoyCaption, or caption by hand. That's the shortlist.

The inputs that matter

• model_name - eight choices, and the default tells you something: MiaoshouAI/Florence-2-base-PromptGen-v2.0. The plain microsoft/Florence-2-* variants are the originals; the PromptGen finetunes (by MiaoshouAI) are retuned specifically for image-generation captions and tags, which is why the pack defaults to one. For SD/Flux dataset work, PromptGen v2.0 is a genuinely good default. Reach for a microsoft/Florence-2-large if you want the vanilla model's more literal descriptions; step down to a base if you're squeezing VRAM (you won't be, these are small).
• precision (fp16 default, or bf16 / fp32) - leave it on fp16. Bump to fp32 only if you hit NaN / garbage output on some hardware; it doubles memory but forces valid numbers.
• attention (sdpa default, flash_attention_2, or eager) - leave this on sdpa. flash_attention_2 is faster but requires the flash-attn package built for your GPU, which most people don't have installed - pick it without that and you get an import error. sdpa is the safe, works-everywhere choice; eager is the slow fallback if sdpa ever misbehaves.

The output is a single model (type MIE_FLORENCE2_MODEL) that feeds Florence2 Describe Image or the batch Caption Images Under Directory. It won't plug into the Janus nodes - different type.

How to install it

Part of the ComfyUI-CaptionThis pack:

• ComfyUI Manager (recommended): search CaptionThis, Install, restart.
• Manual: cd ComfyUI/custom_nodes && git clone https://github.com/MieMieeeee/ComfyUI-CaptionThis, then restart.

Florence-2 weights are not bundled. On first run the node downloads the chosen variant from HuggingFace, or you can grab them by hand - the microsoft/Florence-2-* and MiaoshouAI/Florence-2-*-PromptGen-* repos - and drop each in its own folder under ComfyUI/models/LLM/ (e.g. ComfyUI/models/LLM/Florence-2-base-PromptGen-v2.0/). Behind a slow or blocked connection, set HF_ENDPOINT=https://hf-mirror.com before launch.

Common issues & troubleshooting

Import error mentioning flash_attn / flash attention. You selected flash_attention_2 without the flash-attn package built. Switch attention back to sdpa. This is the single most common Florence-2 stumble.

dtype errors or black/NaN output. A precision mismatch on your hardware. Try precision: fp32 - slower and heavier, but it produces valid numbers where fp16 sometimes doesn't.

First run stalls. It's downloading the model, not frozen. The models are small so it won't take long; watch the console.