ComfyUI Node

Caption Model List

A curated dropdown of image-captioning models — and why the default is the weak one

By mediocreatmybest·Created about a year ago·Updated 12 months ago· 3
Caption Model List
    • selected_model
    model_nameSalesforce/blip-image-captioning-base

    Model Loader's model_name_or_path field is free text - you can type any Hugging Face repo id into it. Caption Model List exists so you don't have to go find one yourself: it's a curated dropdown of fifteen known-working image-captioning models, and its output wires straight into that field.

    What's actually in the list

    The visible options span three model families. BLIP (Salesforce/blip-image-captioning-base and -large) - the default, and small: a dedicated captioning head rather than a full language model. BLIP2 (five variants: blip2-opt-2.7b, a fp16-sharded community rehost of the same, blip2-opt-2.7b-coco, blip2-opt-6.7b, blip2-opt-6.7b-coco, plus flan-t5-backed variants) - meaningfully heavier, since these pair the vision encoder with a real 2.7B or 6.7B-parameter language model backbone, which is where the caption quality improvement over plain BLIP actually comes from. And GIT (microsoft/git-base, git-large-coco) - Microsoft's GenerativeImage2Text line, sitting between the two in both size and quality. The full list runs to fifteen; three more sit past what's shown here, reachable the same way once you open the dropdown in ComfyUI.

    The thing worth knowing before you use the default

    The default selection, Salesforce/blip-image-captioning-base, is also Model Loader's own hardcoded default - so if you never touch either node, BLIP-base is what captions your dataset. Worth knowing: the LoRA-training community's settled opinion on BLIP is not kind. It's been called out repeatedly as producing formulaic, inaccurate captions, and the standing advice - old advice, still repeated - is to hand-caption instead or use a genuinely current model. The two models that actually get recommended today for natural-language captioning are JoyCaption and Florence-2. Neither is on this list.

    That doesn't make this node useless - the BLIP2 and GIT variants here are real steps up from stock BLIP-base, and if you're staying inside this pack's generic Model Loader → Caption Generator path, reaching for a BLIP2-opt-6.7b instead of the default is a genuine quality upgrade for the cost of more VRAM (pair it with Model Loader's use_bitsandbytes quantization if that's tight). But if caption quality is the actual priority, the better move within this same pack is to skip this list entirely and use the dedicated Florence-2 Node, which defaults to a current, well-regarded model this dropdown doesn't offer.

    The inputs and outputs that matter

    • model_name (enum, default Salesforce/blip-image-captioning-base, the field's own tooltip: "List of image captioning models.") - the whole node.

    Output is selected_model (STRING) → Model Loader's model_name_or_path.

    Installing it

    Through ComfyUI Manager: search ComfyUI-Transformers-Pipeline, install, restart. By hand:

    cd ComfyUI/custom_nodes
    git clone https://github.com/mediocreatmybest/ComfyUI-Transformers-Pipeline
    

    then restart. This node itself downloads nothing - picking a model here just produces a string; the actual weights come down when Model Loader runs with that string.

    Common issues & troubleshooting

    The model you want isn't in the dropdown. This node is a convenience, not a limit - type the Hugging Face repo id directly into Model Loader's model_name_or_path field instead. Anything transformers.pipeline() can load will work there whether or not it's in this list.

    Switched to a BLIP2 variant and now it's slow or OOMs. The opt-6.7b variants are genuinely large language models bolted onto a vision encoder, not a small tweak on BLIP-base. Turn on use_bitsandbytes (4bit or 8bit) over on Model Loader before assuming your hardware can't handle it.

    Captions still look weak after picking a "better" model. Confirm the selection actually reached Model Loader - this node only outputs a string, it does nothing on its own, so a broken wire between the two leaves you silently back on Model Loader's own BLIP-base default.

    CategoryTransformers Pipeline

    Inputs (1)

    NameTypeDefaultDescription
    model_nameCOMBOSalesforce/blip-image-captioning-baseList of image captioning models.

    Outputs (1)

    NameTypeDescription
    selected_modelSTRING