llama.cpp Caption Batch

The job it does

You're building a LoRA dataset. You have forty screenshots, sixteen character shots, or a folder of style references, and you need one caption per image - written by something that can actually look at the picture. That's the whole reason this node exists, and it's a narrower job than it looks.

Here's the trap the node is built around. If you feed a batch of images into one chat request, you get one reply describing the batch: "the first image shows…, the second shows…" - and if you instead loop a chat model over images, every caption inherits the previous one's phrasing because the conversation is still open. Both produce caption files that look fine and train worse. Caption Batch sends one independent request per image, in tensor order, with no history carried between items. Image 7 doesn't know what image 6 said.

The result is a native Comfy list of strings, one per image, in order - which is the shape you actually need if you're feeding a caption writer or a training-prep node downstream.

Grounding: what a caption is even for

Natural-language captions are the right format for LLM-encoded bases - Flux, Z-Image, Qwen-Image, Krea 2 - while Illustrious/Pony/NoobAI want Danbooru tags and belong with WD14. The rule that survived the architecture shift is describe what you want to stay adjustable, leave what should be permanent undescribed. A vision model will cheerfully ignore that and describe everything, which is why the durable workflow is auto-caption the bulk set, then hand-audit a small character set. No VLM has solved multi-subject attribution; two people in frame and it will mix up who's wearing what.

So put your actual instruction in prompt - "one paragraph, plain description, no preamble, no lists" - and expect a few re-runs before the output is dataset-grade.

The inputs that matter

images is required: a batch of 1 to 32 images. Behind it, the node slices each frame out before converting it to CPU, so a big batch doesn't explode RAM on the way in.

prompt is the shared instruction, and there are three optional JSON fields that turn the scalar defaults into explicit per-item mappings. For two images:

ids_json:    ["front", "back"]
prompts_json: ["Describe the visible object.", "Describe the visible colors."]
seeds_json:  [101, 202]

Leave them blank and you get zero-based IDs "0", "1"… and every item uses the same prompt and seed. Fill them and they must be exact-length - lists never repeat their last value to pad a short one, which is the behaviour you want for captions and the opposite of what core Comfy list mapping does. IDs are preserved exactly and can be filenames, since the node will not infer filenames from an IMAGE tensor. If you need 00042.png to line up with its caption, say so in ids_json.

Also worth setting: max_tokens (a couple hundred is plenty for one caption, and 2048 is generous), system_prompt for your standing style rule, and release_after_generation - wait for the managed runtime to actually release before returning, which you want if a diffusion step follows in the same queue.

Outputs

captions is the aligned STRING list. result_json is one group record whose rows each carry item_id, seed, state, response, thinking, a nested generation result and any error. The group also has its own release block, kept separate from the per-item results because they're different claims.

That result_json is not decoration. Every row's state is complete, failed or not_attempted, and it's the only way to know which caption is real. The docs say it plainly and it's worth repeating: never drop the blank entries and then zip the remaining captions with the original IDs. That silently shifts every filename after the first failure. Export the group JSON, or check states before writing sidecar files.

The default raise_error policy fails the whole group if any item fails, and exposes no caption list from that run - strict, and correct for a dataset job where a missing caption is worse than a failed run. return_marked_partial gives you the list with blanks in the failed slots plus full state in the JSON, which is fine as long as you read the JSON first.

One shared deadline (request_timeout) covers image preparation, all 32 requests and the release wait, so a 32-image batch needs a much bigger number than a single generation.

Installing and running it

Install the pack - Manager search "comfyui-llamacpp", or comfy node install comfyui-llamacpp - then bring your own llama-server binary and drop a GGUF plus its matching mmproj projector into ComfyUI/models/LLM/gguf/ (a subfolder per vision model keeps things sane). Nothing about the LLM runtime comes through pip; the pack's dependencies are aiohttp, numpy, pillow, psutil and requests.

Then Workflow > Browse Templates > comfyui-llamacpp > caption-batch-aligned is a working two-image graph with IDs first/second and seeds 101/202. Load it, pick an installed vision model in the Start Server node, swap in your own images.

Where people get burned

It needs a vision model, and the projector has to match. If Start Server can't prove a projector fits, it asks you to choose one instead of guessing - deliberate, because pairing an mmproj from a different size or architecture gives you confident nonsense. Server Status shows which one got resolved.

Mismatched dimensions. Native ImageBatch resizes the second image; if your source files differ in size, that resize happens before captioning and the model sees something you didn't inspect.

It writes no caption files. No .txt sidecars here - you get strings and JSON. Pair it with a writer you trust, and give the writer the explicit IDs.

The captioner describes more than you asked for. Narrow the instruction and remember the ceiling: text about images is not a dataset. Audit before it becomes 40 training samples.