Image To Caption

Feed it an image, and it hands that image to OpenAI's GPT-4o or Anthropic's Claude, gets back a written description, and returns it as a string you can save or drop into a dataset pipeline. That's the whole node. It's a thin one, and it knows it.

You reach for this when you're assembling a LoRA dataset and you want natural-language captions of the kind GPT-4o and Claude actually write - no local vision model, no VRAM cost, no weights to download. The readme calls it "a simple node," and that's the honest framing. The flip side is that nothing about the captioning happens on your machine. Every image you put through it leaves your computer and costs you a few fractions of a cent in API billing.

How it works

The mechanism is one function, not a framework. ComfyUI passes the node an IMAGE tensor; the node converts it to a PIL image, resizes it so the longest side is 512 pixels, JPEG-encodes it to base64, and POSTs it to gpt-4o (OpenAI) or claude-3-5-sonnet-20241022 (Claude) together with whatever you typed into Prompt. The model's answer comes back and the node returns it as a single STRING.

That 512px cap is deliberate - it keeps each call cheap and fast. The trade-off: whatever the downscale drops is detail the model never sees. On ordinary scene description that's fine; if your images have small text, logos, or fine details you actually care about captioning, expect the model to blur over exactly the part you wanted.

Two implementation details worth knowing. The node marks itself as always-changed (IS_CHANGED returns NaN), so it re-calls the API on every queue even if nothing changed - re-running means re-paying. And it only handles one image per run; the code asserts on a single frame, so don't feed it a batch.

Inputs and outputs that matter

Four inputs, and only two are yours to actually tune:

• Image - one image at a time, as noted.
• Prompt - the instruction sent to the model. Default is "Describe the image."; something like "Create a concise description for the given image" works better. This is where you encode your captioning rules - be specific about what you want described and what left out.
• API_Key - your key, pasted in. OpenAI key for OpenAI, Anthropic key for Claude. This is the classic trip-up: no key, no output, just errors.
• Service - the pick of the two: OpenAI or Claude.

The one output is Caption (STRING). Wire it into a Save Text node, a String node you reference elsewhere in the graph, or your dataset-prep step.

Installing

Via ComfyUI Manager: search "VLM Captions" or "ComfyUI-VLM-Captions", install, restart. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/5x00/ComfyUI-VLM-Captions

Then restart ComfyUI. The dependencies are openai, anthropic, pillow, numpy, torch, torchvision - nothing you don't already have in a ComfyUI install, and critically no model files. There's no local model in this pack at all; the only real "setup" is holding valid API keys. The repo hasn't been touched since early 2025, so treat it as a set-and-forget utility rather than an actively developed pack.

Where people get burned

• Cost. Every caption is a paid API call, and because the node always re-runs, re-queuing a workflow re-bills the whole batch. A few hundred images adds up quickly.
• Privacy. Images leave your machine for OpenAI or Anthropic. Don't caption anything sensitive, full stop.
• The 512px resize quietly eating the detail you needed.
• Single-image only - batch captioning means looping over images yourself.
• Claude output formatting. The Claude path returns the raw SDK response object rather than extracting just the text, so the caption can come back wrapped in library formatting (TextBlock(...)) instead of a clean sentence. The OpenAI side returns clean text; if you hit the Claude quirk, that's the known rough edge.

And the honest alternative: if the cloud dependence bothers you, the community's default is local captioners - JoyCaption for natural-language captions, Florence 2, or a WD14 tagger when you're captioning for the Danbooru-tagged bases. Caption quality matters more than dataset size, so whichever tool you pick, be deliberate. This node's edge is narrow: you specifically want GPT-4o/Claude-grade natural language and don't want to run a VLM. For small jobs where you already pay for those APIs, it's genuinely handy. For big datasets or private material, go local.