PillarJoyCaption

If you train LoRAs, you already know the drill: JoyCaption is the community's favorite captioning model - an uncensored, natural-language vision model that actually writes good training captions, which is why every "how do I caption my Flux dataset" answer points at it. PillarJoyCaption is a ComfyUI wrapper around fancyfeast/llama-joycaption-beta-one-hf-llava, the Beta One build of that model. The genuinely interesting bit is the twist in the pack's pitch: instead of loading an 8B LLaVA into the same process as your sampler and letting it eat VRAM that KSampler wants, it can run the model as a separate HTTP service on another machine. No API key, no cloud - "distributed" here means "another box on your LAN with spare VRAM."

How it works

Under the hood it's a stock LlavaForConditionalGeneration from transformers. The node assembles a caption prompt from three things: a caption type (one of 12 canned instruction templates, from "Detailed Description" to "Danbooru Tag List"), a caption length, and up to three extra option hints. It appends a suffix asking the model to reply in both English and Chinese, runs a standard generate() pass with your temperature / top_p / top_k / max_new_tokens, then splits the reply into its English and Chinese halves. That's why it returns two caption strings instead of one.

The local path loads the model in-process (first run auto-downloads it to ComfyUI/models/LLavacheckpoints/). The remote path POSTs the image and prompt to base_url's /joycaption/generate endpoint - that server is the companion repo, aicoder-max/Pillar_Service.

The inputs that actually matter

• exec_opt - 远程 (remote) or 本地 (local). The big decision. Remote sends the image over HTTP; local loads the model here.
• memory_mode - only used in local mode. 4-bit ≈ 4.2 GB, 8-bit ≈ 8.5 GB, default ≈ 17 GB. Start at 4-bit; it's a captioner, not a painter, you won't miss the precision.
• caption_type - the 12 presets. For anime-model tags use the Danbooru/e621 tag lists; for Flux-style natural-language captions use Detailed Description; "Stable Diffusion Prompt" and "MidJourney Prompt" produce prompt-shaped output.
• extra_option1–extra_option3 - stackable instruction hints (mention lighting, camera angle, watermark, "refer to people as {name}"). Pick one or two; piling on all three makes the model chattier, not better.
• person_name - fills in the {name} placeholder if you used that extra option.
• max_new_tokens, temperature, top_p, top_k - the usual sampling knobs. 512 tokens, temp 0.7, top-p 0.9 are fine defaults.

Outputs: query (a readable log of what you asked for), en_caption, cn_caption - wire en_caption into a text preview node, a Save Text node, or straight into a caption field for a training run.

Installing it

Easiest is ComfyUI Manager - search Pillar_For_ComfyUI and hit install. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/aicoder-max/Pillar_For_ComfyUI
cd Pillar_For_ComfyUI
pip install -r requirements.txt   # on Windows Portable, use ..\..\..\python_embeded\python.exe -m pip

Heads-up: the requirements pull in transformers, torch, fastapi, and friends - and the 4/8-bit modes need bitsandbytes, which isn't in that requirements.txt. ComfyUI's bundled Python usually has it; if the quantized modes throw, that's the missing piece.

Where people get burned

• The dropdowns are in Chinese. This is a Chinese-authored pack, so you get 远程/本地 for remote/local and 最大节省 (4-bit) for the 4-bit mode. It's a one-time WTF, then you know them by position.
• Leave base_url alone in remote mode and you get an error string in your caption output, not a crash - "Error: Please provide a valid base_url for remote execution." Confusing the first time.
• First local run looks hung while it snapshots several GB from Hugging Face. Let it cook.
• The model is big. On a 8 GB card, 4-bit + the model download is the price of admission; on 4 GB, go remote or use a smaller captioner entirely.