Nodes/ComfyUI_LayerStyle_Advance/LayerMask: Load Florence2 Model(Advance)
ComfyUI Node Runs on cloud

LayerMask: Load Florence2 Model(Advance)

Load Florence2 Model — Microsoft's do-everything vision model, minus the fuss

By chflame163·Created 2 years ago·Updated 4 months ago· 696
LayerMask: Load Florence2 Model(Advance)
    • florence2_model
    versionbase

    Florence 2 is Microsoft's small-but-capable vision-language model, and it's one of the two standard LoRA-dataset captioners along with JoyCaption - the difference in practice is that Florence 2 tends to be tighter and more clinical (some people find it a bit too eager to describe everything down to a painting hanging in the background), while JoyCaption leans more natural and is the one people reach for when NSFW coverage matters. Both beat BLIP by a wide margin, which the community has considered obsolete for captioning for years now. This node just loads the model; the actual captioning work happens in the companion Florence2Image2Prompt node (and a couple of others in the wider pack that use Florence2 for object detection and segmentation tasks).

    There's exactly one setting: version, an enum with fifteen choices covering the base Florence 2 releases (base, base-ft, large, large-ft) plus a set of fine-tuned variants for specific jobs - DocVQA for document question-answering, SD3-Captioner and several PromptGen versions (base/large, v1.5, v2.0) tuned specifically for generating Stable-Diffusion-style prompts rather than plain descriptions, and a couple of CogFlorence community fine-tunes. If you're just captioning a LoRA dataset, base or large (bigger, slower, generally better) is the standard pick; if you specifically want output shaped like a generation prompt rather than a description, one of the PromptGen variants is worth trying instead. The output is a single florence2_model handle that feeds into whichever downstream Florence2 node you're using.

    Install the pack via ComfyUI Manager (search "ComfyUI Layer Style Advance") or git clone https://github.com/chflame163/ComfyUI_LayerStyle_Advance into custom_nodes/, restart, run the requirements installer. The model itself downloads automatically the first time you run it - no manual step needed for the base versions - though you can also grab it ahead of time from the author's Baidu link into ComfyUI/models/florence2 if you'd rather not eat the download mid-workflow. If you want the Flux-tuned variants (gokaygokay/Florence-2-Flux-Large and Florence-2-Flux), those need to be fetched separately and dropped in the same florence2 folder.

    The one real gotcha with Florence2 in this pack is a transformers compatibility break that shows up as the model failing to run correctly on newer transformers versions. The fix, credited in the changelog to kijai's ComfyUI-Florence2 project, is to update the plugin and then copy modeling_florence2.py and configuration_florence2.py from the pack's own florence2_models folder over the matching files inside ComfyUI/models/florence2 - it's a two-file overwrite, not a full reinstall. Beyond that, the usual suspects apply: if you're pulling a PromptGen or CogFlorence variant and it can't reach Hugging Face, that's a network/firewall issue rather than anything wrong with the node, and setting the HF_ENDPOINT environment variable to a mirror is the documented workaround if you're somewhere huggingface.co is unreachable.

    Category😺dzNodes/LayerMask

    Inputs (1)

    NameTypeDefaultDescription
    versionCOMBObase15 options: base, base-ft, large, large-ft, DocVQA, SD3-Captioner, +9

    Outputs (1)

    NameTypeDescription
    florence2_modelFLORENCE2