JoyCaption

If you're about to train a LoRA, you're about to spend an evening staring at images and typing what's in them. JoyCaption exists so you don't have to. It's a vision-language model that looks at an image and writes the caption for it, and it's been the community's go-to auto-captioner for years - the KB's lora-training essay still calls it the uncensored favorite for training captions, and for a good reason: it writes natural-language sentences, which is exactly what Flux and the newer LLM-encoded models want. Feed it tags and you get tag-model behavior; feed it JoyCaption and you get a caption that sounds like a person wrote it. This particular node is the first-party one, straight from fpgaminer, the JoyCaption author himself, wrapped for ComfyUI.

How it works

JoyCaption is a LLaVA-style model (Llama-3.1-8B plus a SigLIP vision tower) that this pack loads through the Download and Load JoyCaption Model node. What JoyCaption adds is a prompt builder on top: it takes your chosen caption style, target length, and up to five extra instructions, assembles them into a query, sends that plus the image through the model's chat template, and returns the generated text. You're not writing prompts - you're picking from menus that map to well-tested captioning styles the model was trained on.

The inputs that matter

• caption_type - the big one. Twelve presets, from Descriptive and Straightforward (great default for training sets) through Stable Diffusion Prompt and MidJourney (chaotic, fun, sometimes great) to Danbooru tag list, e621 tag list, and friends for the anime/Pony crowd. Pick the style that matches what your base model expects.
• caption_length - "any", a vibe ("very short" to "very long"), or an explicit word count up to 260. For character LoRAs people usually want short and factual; for style or general-purpose, longer gives the model more to learn from.
• extra_option1..5 - 28 optional instructions appended to the query. A few are gold for training: "do NOT include info about people that cannot be changed" stops the model baking fixed identity into your captions, and "if it is a photo, mention the camera and settings" dresses captions the way photorealism recipes want. extra_option1 sits on the node; the rest are tucked under advanced.
• person_name - fills the {name} placeholder if you use the "refer to them as {name}" extra option. Ignore it otherwise.
• max_new_tokens, temperature, top_p, top_k - generation settings, all under advanced. Defaults (512 tokens, temp 0.6, top_p 0.9, top_k off) are sensible. Drop temperature toward 0.3 if you want the same caption for the same image every time, which you probably do for training consistency.

Wiring it up

Two outputs: query (the exact prompt built from your settings - genuinely useful for debugging what the model was asked) and caption (the text). The caption feeds a text-save node when you're writing .txt files next to training images, or any STRING input if you just want to read it or feed it back into a prompt. That's the whole loop: Loader → JoyCaption → save file.

Install and troubleshooting

Install through ComfyUI Manager (search "JoyCaption Nodes") or:

cd ComfyUI/custom_nodes
git clone https://github.com/fpgaminer/joycaption_comfyui

then restart. The model downloads automatically on the loader's first run - about 16GB into ComfyUI/models/LLavacheckpoints/.

The two things that actually bite: batch size. This node only captions one image at a time - feed it a batch and you get back a caption that reads "Error: batch size greater than 1 is not supported." Loop your batch instead. And quantization - if you're using 4-bit mode to fit VRAM, expect the "latest version of bitsandbytes" error and fix it with pip install -U bitsandbytes, then accept slightly muddier captions. For training data, that trade is usually the wrong one. Caption quality is the whole point.