PillarJoyCaptionCustom

The sibling PillarJoyCaption node wraps JoyCaption in a dozen fixed templates - detailed description, Danbooru tags, art critic, and so on. That's great until you want something it doesn't offer, like "describe only the pose and framing" or "list what a detective would notice." That's the whole reason this node exists: it's the exact same JoyCaption model and the exact same local/remote machinery, but with two free-text fields instead of preset dropdowns. Think of it as the "developer mode" version of the caption node.

How it works

Mechanically it's identical to PillarJoyCaption - it runs fancyfeast/llama-joycaption-beta-one-hf-llava through a standard transformers generate() call and asks for a bilingual (English + Chinese) reply. The only difference is upstream of the model: instead of picking from canned templates, the node takes your raw system_prompt and user_query and sends them straight through. You're effectively doing prompt engineering on a vision LLM, so specificity pays off - "describe the clothing, pose, and lighting" beats "describe this image."

It shares the same core setting: exec_opt (远程 remote / 本地 local), base_url (only meaningful in remote mode - point it at a Pillar_Service instance), and memory_mode for local runs. The three memory options are the same ladder: 4-bit ≈ 4.2 GB, 8-bit ≈ 8.5 GB, default ≈ 17 GB. First local run still auto-downloads the model to ComfyUI/models/LLavacheckpoints/, so it doesn't pay that download twice if you already ran the other caption node.

The inputs that matter

• system_prompt - the model's role. The default is a generic helpful-assistant line; swap it for "You are a strict Danbooru tagger" or "You are a cinematographer describing shot composition."
• user_query - the actual instruction about this image. Multiline, so you can write real sentences.
• memory_mode, max_new_tokens, temperature, top_p, top_k - same as the preset node. Defaults (512 tokens, 0.7, 0.9) are fine.

Outputs: query (echoes your user query back), en_caption, cn_caption (the English and Chinese halves of the model's reply). Wire en_caption into a preview or Save Text node.

Installing and gotchas

Same install as the rest of the pack - ComfyUI Manager, search Pillar_For_ComfyUI, or git clone https://github.com/aicoder-max/Pillar_For_ComfyUI into custom_nodes and pip install -r requirements.txt. The UI labels are Chinese (远程/本地, 最大节省 (4-bit)), leaving base_url at its default in remote mode dumps an error string into the caption outputs, and if the quantized memory modes throw, bitsandbytes isn't installed. If you've already used PillarJoyCaption, none of this is new - this node is just the same engine with the guardrails off.

This is the node you'll actually reach for once you know what you want out of captions. The presets are a fine starting point; the custom version is where you teach the model your captioning style, and it's the one that makes your training captions stop all sounding the same.