SP_FlorenceCaption

Hand-captioning a 20-image character LoRA dataset is a reasonable afternoon. Hand-captioning 200+ images for a style or concept LoRA is a different kind of task entirely, and it's exactly the gap Florence-2 fills - Microsoft's vision-language model, purpose-suited to writing image captions, and one of the two tools the community actually reaches for on modern LLM-encoder architectures (the other being JoyCaption). This node runs it directly inside ComfyUI.

How it works

Feed it an image, pick a model variant, and it returns a caption string. The eleven model choices span stock Microsoft checkpoints (base, base-ft, large, large-ft, and a DocVQA fine-tune) and a handful of community fine-tunes built specifically for this job - CogFlorence 2.1 and 2.2, gokaygokay's SD3 and Flux captioners, and MiaoshouAI's PromptGen v1.5 in base and large sizes. The community fine-tunes generally write more "prompt-shaped" output than the stock checkpoints, which tend to read a bit more clinical - worth trying one of those if the plain Florence-2 captions feel too dry for what you're training toward.

task picks between detailed_caption and more_detailed_caption - Florence-2's two verbosity tiers. Most LoRA training guides actually want the deeper one; plain detailed_caption tends to undershoot for training purposes, so more_detailed_caption is the better starting point unless you have a reason to keep captions short.

The inputs and outputs that matter

• image - what you're captioning.
• model - pick a stock Florence-2 checkpoint or one of the community captioning fine-tunes.
• precision - fp16, bf16, or fp32. bf16 is generally the safer default on modern GPUs; reach for fp32 if you hit odd numerical instability.
• task - detailed_caption vs more_detailed_caption.
• seed - for reproducibility across runs.
• Output: caption (a list) - running this over a batch of images returns one caption per image, so you can pipe an entire dataset folder through in a single pass instead of one image at a time.

How to install it

Through ComfyUI Manager: search "SP-Nodes," install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/bananasss00/ComfyUI-SP-Nodes

Restart ComfyUI. The first time you run this node with a given model selected, expect it to pull that checkpoint down from Hugging Face - anywhere from a few hundred MB to over a gigabyte depending on which variant you pick - so the first caption pass will be noticeably slower than every one after it, once the model is cached locally.

Common issues

If a model choice fails to download, that's almost always a Hugging Face Hub connectivity issue rather than anything wrong with the node - retry, or check whether the specific repo needs you to be logged in. If your captions come back reading shorter or vaguer than you expected, double-check task is actually set to more_detailed_caption rather than the plain variant; it's an easy setting to leave on the default and then wonder why the output feels thin. And per general captioning practice: even a good auto-caption benefits from a manual pass on a small, high-value dataset - Florence-2 gets you most of the way fast, not necessarily all the way there.