Nodes/ComfyUI-WepeNerd/Image Captioner
ComfyUI Node

Image Captioner

Caption a whole batch locally, without touching an API

By WepeNerd·Created 5 months ago·Updated about 17 hours ago· 0
Image Captioner
  • model
  • image
  • captions
style
instruction

Image Captioner reads an image and writes a caption about it - the VLM half of the WepeNerd Local AI workflow. Wire Local AI Model's model output into it, connect an IMAGE, and you get caption text out. It's the node you reach for when you're captioning a LoRA training set, or when you want to seed an img2img or image-to-video prompt from an existing frame without describing it by hand.

The architecture note that decides everything: captioning needs a vision projector. The Local AI Model node's projector input has to point at a real mmproj file (like mmproj-model-bf16.gguf), not Auto / None. Without one, the node errors - the model can't see the image, and no amount of prompt tweaking fixes that. Projectors are model-specific too: the README warns that discovery is not compatibility, so a projector built for one Qwen variant won't necessarily work on another. If your image request gets rejected, that mismatch is the first thing to check, along with whether your llama.cpp build is recent enough for multimodal image_url support.

The inputs you'll actually touch:

  • style - Dataset, Detailed, Short, Tags, or Custom. The first four map to canned caption instructions; Dataset is the safe default for training captions on natural-language bases like Flux or Z-Image, Tags emits comma-separated booru tags for the Danbooru-trained anime lineage (Illustrious, Pony, NoobAI), and Detailed is for rich img2img seeding.
  • instruction - optional override that replaces the default "describe this image" instruction, or defines your own system prompt entirely when style is Custom.

Under the hood it's the same path as the advanced captioner: images are downscaled (never upscaled) to a 1024-pixel max edge and JPEG-encoded at quality 90, then sent to llama-server as data URLs with reasoning disabled. Here's the useful bit for dataset work: the output is a list (captions), one caption per image, and the whole batch runs through a single server acquisition. ComfyUI batches a folder with Load Image Batch or similar, and each image comes back with its own caption - exactly the shape you want for writing .txt caption files next to a training folder.

Two honest caveats, both grounded in how every VLM in this category behaves. First, multi-subject attribution is the universal weakness - with two people in frame, VLMs mix up who's wearing what. Auto-caption the bulk, audit the important stuff by hand. Second, the local-model trade is a real one: a 27B Qwen quant is far more capable than a 3B, and the difference shows up in caption quality more than raw speed. The README's reference model is a big abliterated Qwen GGUF for exactly that reason.

Install comes with the rest of the pack - ComfyUI Manager (search WepeNerd) or clone https://github.com/WepeNerd/ComfyUI-WepeNerd.git into custom_nodes, pip install -r requirements.txt, restart. And don't skip the llama-server step: the model file alone can't run anything.

CategoryWepeNerd/Local AI

Inputs (4)

NameTypeDefaultDescription
modelGGUF_LLM_CONFIG
imageIMAGE
styleCOMBO5 options: Dataset, Detailed, Short, Tags, Custom
instructionoptSTRING

Outputs (1)

NameTypeDescription
captionsSTRING