OpenAI GPT4V 🧅

Every now and then you have an image and you need to know what's in it - not a CLIP score, not a tag list, but an actual sentence describing the scene. This node sends your IMAGE to a GPT-4 vision model and returns the answer as a STRING. It's the "describe this picture" node, and it plugs into your graph like any other text producer.

How it works

Under the hood it does exactly what you'd expect: takes the first image in the batch, saves it as a JPEG, base64-encodes it, and POSTs it to OpenAI's chat completions endpoint with a vision payload. Your prompt (default "What's in this image?") is the instruction, and the detail setting (auto/low/high) controls how much visual resolution the API uses - high is more accurate for fine detail but slower and pricier, low is fine for a quick scene summary.

The model dropdown defaults to gpt-4o and also lists gpt-4o-mini, gpt-4-turbo, and gpt-4-vision-preview. Same caveat as the pack's GPT node: this is a frozen dropdown, so retired model IDs will error until the pack updates. max_tokens caps the response length, and there's a seed input that the code reads but genuinely ignores (vision calls don't take a meaningful seed) - wire a Seed Generator in for consistency if you like, but don't expect it to change anything.

Inputs and outputs

• image - the image to inspect (first of the batch).
• prompt, model, detail, max_tokens, seed.
• Output: a single STRING with the model's description.

One quirk: the image is written to a file named gpt4v_tmp.jpg in the working directory, then deleted on the next run. Not a problem, but if you see a stray gpt4v_tmp.jpg appear, that's this node - it's transient.

Where you'd actually use it

Auto-captioning is the killer app: feed a folder of generated images through it, get a natural-language description for each, and use that text to build better prompts, organize results, or drive a loop where the model critiques its own output. Combined with the pack's GPT node, you can build a full evaluate-and-regenerate pipeline - ask GPT4V what's wrong, have GPT rewrite the prompt, feed it back into the sampler. That loop is genuinely powerful, and it's the direction a lot of serious workflows went.

Installing it

It's in the ComfyUI-NegiTools pack - ComfyUI Manager (search "ComfyUI-NegiTools") or:

cd ComfyUI/custom_nodes
git clone https://github.com/natto-maki/ComfyUI-NegiTools
pip install -r ComfyUI-NegiTools/requirements.txt

then restart with OPENAI_API_KEY set before launching ComfyUI. No key, no calls - the node will fail fast at authentication.

The honest take

For a quick "describe this image" job, this is the easiest path in the graph. Just don't expect deep analysis or consistent opinions - GPT-4V is chatty and sometimes confidently wrong about details, so treat its output as a draft to be used, not truth to be trusted. And remember every call costs API money, so for bulk captioning a local VLM node will eventually be cheaper - but as an occasional tool, this one is hard to beat for convenience.