✨Image Caption (VLM)

You have a reference image and no idea what prompt made it. ✨Image Caption (VLM) is the modern answer: it hands the image to a vision-language model and gets back a text description you can use as a prompt. It's the LLM-era upgrade on the old CLIP interrogator / WD14 tagger trick - instead of fuzzy tag guesses, you get actual sentences, or actual tags, depending on which rule you pick. Same pack, same deal as the rest: it's an API client. No local model, no downloads.

This is a workhorse node. Reverse-engineering prompts from reference art, auto-captioning datasets, even generating image-to-video prompts from a starting frame - it all routes through "show the VLM a picture, get text back."

How it works

The mechanism is simple and honest: the image gets base64-encoded to JPEG and sent to the chosen VLM along with a system rule and your optional user_prompt. The rule dropdown is where the magic lives, because the shipped presets are targeted at different destinations:

• 像素级描述 (pixel-level description) and Detail Caption - natural-language prose, the right choice for Flux/Qwen-class caption-trained models.
• 图像描述-Tag风格 and Caption-Tags - Danbooru-style tags, which is what Illustrious/NoobAI and the rest of the SDXL anime lineage actually wants.
• 图像编辑重绘 and Qwen-Image-Edit指令优化-视觉版 - produce edit instructions rather than captions, for instruction-based editors like Qwen-Image-Edit.
• i2v物理动态提示词 - image-to-video prompt generation for Wan, Kling, Sora-class models.

If none fit, flip custom_rule and paste your own system prompt.

Batch input is where the node earns its keep. Feed it an IMAGE batch and it processes every frame as a separate VLM call, joins the results with \n---\n into the caption_text output, and also exposes each frame's caption individually through caption_list. For captioning a training set frame-by-frame, that list output is exactly what a captioning script wants.

The inputs that matter

• image - a single image or an IMAGE batch. It errors politely if you connect nothing.
• rule - the paradigm selector described above. The default 像素级描述 is prose; switch to a tag rule for anime SDXL models.
• user_prompt - extra instructions sent alongside the rule, e.g. "describe the lighting in detail."
• vlm_service - 智谱/glm-4.6V-Flash by default, or your Ollama model for a local run.

Outputs are caption_text (STRING, all frames joined) and caption_list (STRING list, per-frame). Wire caption_text into a prompt, or caption_list into whatever downstream node consumes a list.

Install

Manager → search Prompt Assistant, install, restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/yawiii/ComfyUI-Prompt-Assistant.git

Then configure a service in the pack's settings - Zhipu's free GLM-V flash models work well enough to start, and Ollama is there if you want it all local.

Where people get burned

Two things. First, a batch of N images means N separate API calls - the node doesn't batch them into one request. Your free tier quota will notice. Second, rule/model mismatch: feeding a Tag-style caption to a Flux model (or a prose caption to an anime SDXL model) quietly degrades the output rather than failing loudly. Caption quality genuinely matters more than dataset size for training, per every trainer who's been burned by it, so pick the rule that matches what you're captioning for.