Image Caption Node

Drop in an image, and this node writes the prompt for you - and not just as text. It also hands you a finished CLIP conditioning, ready to plug straight into a KSampler. No API key, no cloud call, no web UI to babysit. Everything runs through your local HuggingFace transformers install, the same way the models you already load do.

Let's be clear about what this isn't, first. This is a small, barely-known pack (this page has had 32 impressions total), and its captioner is BLIP/GIT-based. The community is blunt that BLIP is weak - in LoRA-training circles it's "widely considered unusable" for captioning datasets, where the modern picks are WD14 Tagger for Danbooru tags and JoyCaption or Florence 2 for natural-language captions. So don't point this at a training set. Do reach for it when you want to reverse a reference image into a starting prompt without hand-writing a description.

How it works

The node loads a HuggingFace-format model from ComfyUI/models/image_captioners/, where each subfolder is one captioning model. On first run the pack creates that folder for you. When you queue, it reads the model's config.json to figure out the architecture, loads it with transformers on your torch device (bf16/fp16 if ComfyUI says your card can, fp32 otherwise), runs the image through the model's processor, and calls model.generate() with the knobs below.

The clever bit is what comes out. Instead of just a string, it also tokenizes the caption with your clip input and returns a real CONDITIONING. That's the thing most caption tools - and every WD14-style tagger - won't hand you. Wire it into the positive input of a sampler and your prompt is already encoded, no CLIP Text Encode needed.

The inputs that actually matter

• clip - any CLIP model, typically the same one your checkpoint uses. It's only used to encode the caption, so pick the one that matches your target model.
• image - the reference image, uploaded via the node.
• model_name - the folder under models/image_captioners you downloaded.
• max_new_tokens / min_new_tokens - caption length. Defaults are 50/20. The README's one actual tip: raise both for more accurate captions.
• num_beams - beam search width. Default is 20, which is a lot; paired with penalty_alpha (default 0.6) this is contrastive-search decoding, so expect a caption to take a few seconds even on a decent GPU.
• preprocess_mode - how much cleanup the caption gets before output: none (raw), exact_prompt (drops exact duplicate lines), or exact_keyword (dedupes and cleans tags so things like ((masterpiece:1.2)) come out sane). For tag-style SDXL models, exact_keyword is usually right.

top_k and repetition_penalty are there if you need them; the defaults are fine.

Outputs

• clip_output (CONDITIONING) - caption encoded with your CLIP. Feed it to a KSampler.
• string_output (STRING) - the plain caption, if you want to edit it or push it through other text nodes first.

Installing

The easy way is ComfyUI Manager - search for "image-caption-comfyui" and install. Or clone it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/alpertunga-bile/image-caption-comfyui

Then restart ComfyUI. The one dependency is transformers (the pack's pyproject.toml declares nothing else). Models are not auto-downloaded - grab the pretrained BLIP or GIT model from the pack's Google Drive link and unzip it so each model sits in its own folder under ComfyUI/models/image_captioners/. Then restart again and pick it from model_name. The two offered models are both marked "Training In Process," which is the author's way of saying these are off-the-shelf HF checkpoints, not finely-tuned ones.

Common issues

• Import errors or weird model behavior → the README's answer is a transformers version problem. Upgrade it: pip install --upgrade transformers in your ComfyUI environment (for the Windows portable build, that's python_embeded\python.exe -s -m pip install --upgrade transformers).
• model_name is empty → the model folder isn't under models/image_captioners. Restart after placing it.
• A saved workflow breaks after an update → the author warns the node's variables have changed between releases; deleting the node and adding it fresh usually fixes it.
• First run creates models/image_captioners - that's expected, not an error.

The honest verdict

This is a neat zero-key way to auto-prompt, and the direct conditioning output genuinely sets it apart. But it's a hobby pack running an outdated captioner - treat the output as a starting point, not a caption for a training set. For quick reference-image-to-prompt loops, though, it's exactly the right shape: image in, prompt and conditioning out.