Image to Text - Auto Caption

This node takes an image in and hands you back a text description - one node, no separate captioning script, no leaving ComfyUI. Under the hood it's really three captioners wearing one trenchcoat: BLIP, LLaVA-1.5-7B, and MiniCPM-V-2, and you pick which ones actually run with a set of checkboxes. Point it at a render, flip on a model, and you get a caption string you can feed straight back into a prompt - handy for automating an img2img loop, or for getting quick text descriptions of objects in a scene to feed into a segmentation node for object removal.

Where it actually earns its keep

Be honest about what this isn't: if you're captioning a LoRA training set, this isn't the tool. BLIP in particular has a rough reputation there - the community consensus is blunt about it being outdated and formulaic, and the standard advice for training data has moved to JoyCaption or Florence 2 for natural-language captions, or WD14 for Danbooru-tag targets. That's not a knock on this node, just a different job.

Where this node earns its keep is inside the graph, at generation time. Feed it an output image and use the caption to auto-write your next img2img prompt, or ask "what objects are in this image" and pipe the answer into a segmentation node to find and mask things by name. It's a live description tool, not a dataset-captioning pipeline - "what's in this frame, right now, in text," not "build me 10,000 clean captions."

How it works

BLIP is the fast path: toggle use_blip_model on (it's on by default) and it spits out one short caption, optionally with a prefix you supply. LLaVA and MiniCPM are the deeper path - they're actual vision-language models, so instead of a single caption they answer a list of questions you feed them, one per line, and hand back the combined answers as your caption text. You can flip on more than one model at once (or use use_all_models to run all three), and it just concatenates whatever each one produces.

The default question list is worth looking at, because it doubles as a decent txt2img prompting checklist straight from the pack author: subject, medium, style, influential artists, sharpness/detail, environment, objects, composition, color, and lighting. Ask those ten questions about an image and you basically get a structured breakdown of what a good prompt for it would contain - which is the whole point if you're trying to reverse-engineer a look.

Inputs and outputs that matter

• input_image (required, IMAGE) - the only required field.
• use_blip_model / use_llava_model / use_mini_pcm_model / use_all_models - which captioner(s) run. BLIP is on by default; the others you turn on deliberately, mostly because they're much bigger downloads.
• blip_caption_prefix (default "a photograph of") - text BLIP's caption gets appended to. Change it if you're not captioning photos.
• prompt_questions (multiline) - the question list LLaVA/MiniCPM answer, one question per line. Rewrite this to ask about exactly what you care about instead of the default ten.
• temperature, repetition_penalty, search_beams, min_words, max_words - standard LLM generation knobs for the LLaVA/MiniCPM side. Leave these alone until a specific answer is coming back too short, too repetitive, or too random - then nudge one.
• exclude_terms (default "watermark, text, writing") - terms you'd rather the caption steer away from mentioning.

The single output is caption (STRING) - wire it into a CLIPTextEncode, a text-save node, or whatever's downstream in your automation.

Installing it

Through ComfyUI Manager: search img2txt-comfyui-nodes, install, restart. By hand:

cd ComfyUI/custom_nodes
git clone https://github.com/christian-byrne/img2txt-comfyui-nodes
cd img2txt-comfyui-nodes
pip install -r requirements.txt

Models are not bundled - they download automatically through Hugging Face's cache the first time you actually toggle a model on, so if you never touch LLaVA or MiniCPM, they never download. Sizes: BLIP ~2GB, MiniCPM-V-2 ~6.8GB, LLaVA-1.5-7B ~15GB. That last one is a real download and a real chunk of VRAM if you enable it alongside BLIP - budget for it before you flip the toggle on a slow connection or a tight GPU.

Where people get burned

The MiniCPM dependency pins transformers<=4.41.2 - an upper bound, not a floor. If another custom node pack in the same environment wants a newer transformers, pip will fight you over it; worth knowing before you install a stack of packs into one environment and wonder why one of them silently breaks. LLaVA's requirements (bitsandbytes, accelerate) are the usual quantization stack - fine on Linux, more prone to version mismatches on Windows, so that's the first thing to check if LLaVA errors out before it even gets to inference.

Beyond that: if your caption comes back empty or you get an error about no active model, you likely have every toggle off - BLIP is on by default but it's easy to disable while experimenting and forget to re-enable anything. And if you want Chinese output, MiniCPM handles it natively with no extra configuration; BLIP and LLaVA don't.