CXH_DownloadAndLoadFlorence2Model

This is the Florence-2 loader for the pack - the node that grabs Microsoft's vision-language model, drops it in your models/ folder, and hands you a ready-to-use FL2MODEL for CXH_Florence2Run. Florence 2 is the community's other favorite captioning backbone (the captioning half of the "JoyCaption or Florence 2" advice you'll see everywhere), and unlike JoyCaption it also does OCR, object detection, region captioning, and VQA. One loader, a whole toolbox.

The source header notes it's a fork of un-seen/comfyui-tensorops, kept alive because upstream broke on newer ComfyUI. That's the standard history for this kind of node - the original idea is shared across a dozen packs, this is the version that ships in this one.

What it loads

Three inputs, all dropdowns:

• model - nine choices. The default is MiaoshouAI/Florence-2-large-PromptGen-v1.5, a community fine-tune of Florence 2 trained on CivitAI-style SD prompts. It's the one most people should leave selected: it adds mixed_caption and generate_tags tasks on top of stock Florence 2, which is why it's the pack's default. The stock Microsoft Florence-2-base/large (-ft) choices are there for classic captioning; HuggingFaceM4/Florence-2-DocVQA for document questions; and thwri/CogFlorence-2.2-Large is the CogVLM-based variant the README pairs with MiniCPM.
• precision - fp16 (default), bf16, or fp32. fp16 is the sensible default; bf16 if your GPU prefers it; fp32 only if you're debugging precision artifacts and have VRAM to burn.
• attention - flash_attention_2, sdpa (default), eager. sdpa is the right call: it needs no extra install, and flash attention only matters on large batch work.

The model is downloaded via snapshot_download into models/LLM/<model_name> on first use - same LLM/ folder the JoyCaption Llama lives in, so don't be confused when everything shares one directory. Output: a single FL2MODEL for the run node.

One neat implementation detail: the loader monkey-patches transformers' import hook to skip the flash_attn requirement that Florence 2's modeling file otherwise demands. That's why this pack works without you installing flash-attn - a real papercut it saves you from.

Installing

Pack install - ComfyUI Manager (search "Comfyui_CXH_joy_caption") or:

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

Restart, then pick a model. First load downloads a few GB from the Hub - expect a wait.

Troubleshooting

• First-run download is slow and prints "Downloading … to: models/LLM/…". That's normal; it resumes, not restarts, if interrupted.
• Switching models re-downloads to a separate LLM/<name> folder each time - each variant is its own download, so pick your default and stick with it.
• OOM on an 8 GB card: switch precision to fp16 (it should already be), avoid the large variants unless you need them, and keep the run node's keep_model_loaded off so the model offloads after each inference.
• The loader runs on whatever device ComfyUI's get_torch_device() returns, so this one at least isn't hardcoded-CUDA like the JoyCaption loaders - but you still need enough VRAM for the large variants.