JoyCaption (Custom)

The regular JoyCaption node is a menu. This one is a blank page. Same model, same generation engine - but instead of twelve preset caption styles and a wall of extra options, you type the exact system prompt and user query you want, and the model just answers. It's the node you reach for the moment a preset doesn't fit, or when you want to use JoyCaption as something other than a captioner.

That last bit is the underrated use. JoyCaption Beta One was trained with VQA data on top of its captioning job, which means with the right question it'll tell you what camera was likely used, what's in the background, or whether there's a watermark - the things people usually want out of extra_option style instructions but with far more control. And for training, this is how you implement a captioning style the presets don't cover: you can paste in a style guide, demand very specific formatting, or tell it exactly which tags to include in the output.

How it works

Mechanically it's identical to the standard node: image in, the loader's JOYCAPMODEL in, and the model generates text through its chat template. The only difference is that no prompt is built for you - system_prompt and user_query go straight into the conversation. What you type is exactly what the model sees, so the prompt-crafting skill you'd use with any LLM applies here.

The inputs that matter

• system_prompt - sets the model's behavior. The default is the same uncensored helper line the standard node uses internally; leave it unless you're trying to change the model's attitude or role. This is where you'd say "you are a strict tag extractor" or "you are a product photographer's assistant."
• user_query - the actual instruction, multiline so you can write a small spec. The default, "Write a detailed description for this image," is a fine starting point. Replace it with "Describe only the lighting and camera settings" or "Output a comma-separated list of Danbooru tags" or "List every piece of text visible in the image."
• max_new_tokens, temperature, top_p, top_k - same generation knobs as the standard node, defaults of 512 / 0.6 / 0.9 / off. Lower temperature if you want repeatable output across a dataset.

There's no caption_type, no length picker, no extra options - that's the whole point. You write the spec once, and it applies to every image you feed through.

Wiring it up

A single output: the generated response as a STRING. Same destinations as the standard node - save it next to your training images, pipe it into a prompt, or inspect it. One small trade: you don't get the query echo the standard node outputs, so if you're debugging what the model was asked, remember what you typed - it's exactly what went in.

Install and the gotcha to expect

Same as its siblings - ComfyUI Manager (search "JoyCaption Nodes") or:

cd ComfyUI/custom_nodes
git clone https://github.com/fpgaminer/joycaption_comfyui

then restart and let the loader pull the ~16GB model into ComfyUI/models/LLavacheckpoints/ on first run.

The one gotcha that will bite you regardless of which of the three nodes you're using: it's single-image only, so a batched input returns "Error: batch size greater than 1 is not supported." Loop instead of batching. And if you run it in 4-bit mode and hit the bitsandbytes version error, pip install -U bitsandbytes is the whole fix - though with a custom node like this, where the output quality is on you and your query, you'll want the full-precision model anyway.