ComfyUI Node

Captioner

Dataset captions from a VLM without installing a single local model

By power88·Created 10 months ago·Updated 4 months ago· 0
Captioner
  • client_info
  • image
  • extra_parameters
  • Caption
languageEnglish
unload_model_after_chattrue
num_max_sentences10

If you've ever spent an evening captioning a training set by hand, you know why this node exists. Captioner sends one image to a vision LLM and gets back a natural-language caption, in a language you pick, capped at a sentence count you control. No local caption model to download, no Florence-2 weights, no tagger setup - just an API call. It's the fastest way to batch-caption a dataset when you'd rather be training.

How it works

Feed it a client_info from the API LLM Loader, an image, and a language, and it runs a fixed, well-tuned captioning prompt against a VLM. The system prompt is adapted from the Qwen-Image paper's captioning guidance - describe concretely, include object attributes and relationships, quote visible text verbatim, don't invent anything - which is about the right spec for training captions. The node calls the model with max_tokens of 4096 and returns a Caption string.

Inputs that matter:

  • client_info - from the loader.
  • image - the image to caption. Needs a vision model behind the loader; a text-only LLM can't do this.
  • language - default English. Type any language you want the caption written in; handy if you train multilingual or want non-English style captions.
  • num_max_sentences - default 10, and the author has strong opinions here. His guidance: 2 for CLIP-based encoders (SD 1.5/SDXL, and the CLIP part of ClipTextEncodeFlux), 4 for T5-based or Qwen-VL models (the t5xxl part of ClipTextEncodeFlux, or Qwen-Image), 10 or more for Qwen-Image (the README is more relaxed at 4+, but the node's own tooltip points at 10+). That maps onto how the encoders actually work: CLIP is a short-prompt tag machine, while T5 and the Qwen-class LLM encoders want longer natural language. Note this is a request, not a hard cap - the model treats it as "up to N sentences."
  • unload_model_after_chat - Ollama only; unloads the model from memory after each call (keep_alive: "0"). Leave it on if the same GPU is doing the training.
  • extra_parameters (optional) - wire in the Extra Parameters node if your model supports thinking; the node passes it through as reasoning mode.

Use it for training, with the usual caveats

The KB's training lore still applies: caption the things you want to vary, leave fixed things undescribed, and remember the caption style has to match the architecture you're training for. This node writes descriptive sentences - perfect for LLM-encoder bases like Qwen-Image and the Flux T5 path. For the tagged-anime lineage (Illustrious/Pony/NoobAI), the community's standard is a WD14 tagger producing comma-separated Danbooru tags, and a sentence-style VLM caption is the wrong tool; use this one for the models that actually want prose.

Install and gotchas

Install the pack via ComfyUI Manager (search ComfyUI-PDiD-LLM-Nodes) or clone it into ComfyUI/custom_nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/power88/ComfyUI-PDiD-LLM-Nodes

Restart, and confirm you're on ComfyUI 0.3.51+ (this pack uses the V3 node schema, so older versions won't even show it). There are no model downloads - everything is API. Cost is the real tax: every image is a full vision-model call, so a 10k-image dataset is 10k API calls. If that number makes you wince, point the loader at a local Ollama vision model instead and let it grind through the set overnight on your own hardware.

Where people get burned: using a non-vision model and getting garbage or an error, expecting num_max_sentences to be a hard truncation (it isn't), and - the classic - feeding the output straight into a training run without reading a sample, because the model will occasionally invent or smooth over details that matter for what you're training.

CategoryLLM

Inputs (6)

NameTypeDefaultDescription
client_infoCLIENT_INFOThe LLM client info.
languageSTRINGEnglishThe language of the caption.
unload_model_after_chatBOOLEANtrueWhether to unload the LLM model after the chat. Only Ollama is supported.
num_max_sentencesINT101–100The maximum number of sentences for the caption. Use 2 for CLIP model. Use 4 for FLUX. Use 10 and more for Qwen-Image.
imageIMAGEThe image for the LLM model.
extra_parametersoptEXTRA_PARAMETERSThe extra parameters for the LLM model.

Outputs (1)

NameTypeDescription
CaptionSTRING