🦙 LlamaCpp Caption

Let's get the biggest misconception out of the way first: the pack is called "autocaptioner," but it doesn't caption anything on its own. 🦙 LlamaCpp Caption is a bridge. It takes an image from your ComfyUI graph, POSTs it to a llama.cpp server you're already running with a vision model loaded, and drops the model's description back into your workflow. No cloud, no API key (the code sends a dummy sk-no-key-required header that llama.cpp happily ignores), no per-call fees. If you've been eyeballing the "run your own local VLM for captioning" path and wanted it inside ComfyUI instead of a separate script, this is the missing half-inch of plumbing.

It's also one of the most readable node packs you'll install - the whole thing is a couple hundred lines of MIT-licensed Python, single commit, June 2026. Read it before you trust it; it takes ten minutes and you'll understand the entire pack afterward.

How it works

The mechanism is boring in the best way. The node converts your IMAGE tensor to a base64 JPEG (quality 90), stuffs it into a standard OpenAI vision-format message - text plus an image_url data URI - and fires it at {server_url}/v1/chat/completions. That endpoint is llama.cpp's OpenAI-compatible server API, so any vision GGUF that works there works here: Qwen-VL, Gemma 3, LLaVA, and the one everyone actually wants, JoyCaption Beta One (people run it as a Q8 GGUF through llama-server exactly like this). It parses choices[0].message.content, then runs a regex pass that strips <think>, <reasoning>, and <thought> blocks.

That last bit is the genuinely thoughtful touch. Reasoning VLMs like Qwen3-VL-Thinking love to emit scratch-work wrapped in those tags, and if it leaked into a training caption it'd poison your LoRA set. The strip is case-insensitive and multiline, and it collapses the leftover whitespace. If your model doesn't wrap its reasoning in those tags, though, you're on your own - the regex only knows those three patterns.

The inputs that matter

Four required inputs, two optional. The required image tensor is self-explanatory - it's the picture being captioned - and the rest are:

• server_url - default http://localhost:8080, which conveniently matches llama.cpp's default server port. Change it only if you ran llama-server with --port elsewhere.
• prompt - default "Describe this image in detail." This is your caption style dial; tune it per caption mode.
• temperature - 0.0–2.0, default 0.7. Lower it toward 0.2 if you want consistent captions across a batch; that's usually what you want for training data.
• max_tokens - 64–8192, default 1024. Plenty for most captions; bump it if you're using a long-format mode.
• system_prompt (optional) - defaults to an image-captioning-assistant persona. This is where you can drop in a JoyCaption-style "describe exhaustively, attribute everything" instruction.
• image_path (optional) - a passthrough from JoyCaption's Image Batch Path, handed straight to CaptionSaver.

Outputs are two strings: caption and the same image_path, passed through untouched so CaptionSaver can write <filename>.txt beside each image. That's the whole JoyCaption-compatible contract: Image Batch Path → 🦙 LlamaCpp Caption → Caption Saver. Batch works because Image Batch Path outputs lists and ComfyUI auto-iterates, so each image gets its own request to the server.

Installing and wiring it

The README's install path is developer-flavored - symlink or copy the folder into custom_nodes/, then:

cd ComfyUI/custom_nodes
git clone https://github.com/Pranjwal-Jha/llamacpp-comfyui-autocaptioner
pip install requests Pillow numpy

That's the entire dependency list - requests, Pillow, numpy, and torch is already ComfyUI's. You may also find it under "llama-api-comfy" in ComfyUI Manager, but given how fresh this pack is, don't be surprised if it isn't indexed yet. Then the real setup, the part the README can't do for you: a running vision server.

# Either a GGUF that bundles its projector:
./llama-server -m your-vision-model.gguf

# ...or a separate mmproj:
./llama-server -m your-model.gguf --mmproj your-mmproj.gguf

# ...or straight from Hugging Face:
./llama-server -hf ggml-org/gemma-3-4b-it-GGUF

The honest framing: this node is for people who already live in llama.cpp-land, or want to. If you'd rather not run an external server at all, Ollama-based nodes and in-graph GGUF loaders do the same job with a different setup cost. And remember you're budgeting VRAM for two models now - a quantized 4–8B VLM beside your diffusion model, not the 27B one you've been daydreaming about.

Where people get burned

Most failures are "the server isn't where I said it was." Wrong port, server never started, model without a projector - you'll get a connection error or a llama.cpp error string, and the node's error messages are actually decent at telling you which. The silent one is VRAM: an 8B vision model plus your checkpoint on one card gets you OOMs that look like ComfyUI's fault. Drop the quant, not the workflow. Vision models also want real context - 6–8K isn't excessive - and if captions come back truncated, raise max_tokens, not context. And as with any fresh pack, trust-but-verify: caption a few images by hand before you let it loose on a thousand-image dataset, because multi-subject attribution is where every VLM captioner still trips up.