LayerUtility: JoyCaption2(Advance)
JoyCaption2 — the original JoyCaption-alpha-two captioner, wired for ComfyUI
- image
- extra_options
- text
This is JoyCaption's earlier "alpha-two" generation doing actual captioning work - everything the loader node (Load JoyCaption2 Model) sets up gets used here. fpgaminer built the whole JoyCaption line to solve one specific problem: every good vision-language model at the time was either expensive and censored (ChatGPT) or open but weak and still censored on anything past PG (CogVLM). JoyCaption trades some raw benchmark polish for being genuinely uncensored and free, which is exactly why it became the default recommendation for captioning LoRA training sets rather than something like BLIP, which the community wrote off years ago as producing formulaic, inaccurate captions.
Unlike the newer Beta One node, this one bundles the model config directly rather than taking a pre-loaded handle for everything - you still wire in llm_model, device, dtype, and vlm_lora here even with a separate loader existing, and there's a use_global_model toggle specifically for sharing a model already loaded elsewhere in your graph instead of loading a fresh copy. caption_type gives you nine formats: Descriptive, a looser Descriptive (Informal), Training Prompt (shaped specifically for feeding back into diffusion training, which is the whole reason JoyCaption's training-prompt mode exists as a distinct format from plain description), MidJourney-style, two Booru-style tag list formats, and three non-training formats (Art Critic, Product Listing, Social Media Post) for when you just want a description rather than dataset captions. caption_length controls verbosity from any/very short up through numeric targets. user_prompt, if filled in, overrides caption_type and any extra_options entirely - use it for full manual control. top_p, temperature, and max_new_tokens are standard sampling controls. cache_model keeps the model resident between runs rather than reloading it each execution, which is worth turning on if you're captioning a batch and don't want the load cost paid repeatedly. The optional extra_options input takes a JoyCaption2ExtraOption handle for fine-grained control over what gets included or excluded (lighting, camera angle, watermark mentions, and similar). Output is text.
Install the pack through ComfyUI Manager (search "ComfyUI Layer Style Advance") or git clone https://github.com/chflame163/ComfyUI_LayerStyle_Advance into custom_nodes/, restart, run the requirements installer. This node needs three separate model pieces in place: the LLM in ComfyUI/models/LLM, the SigLIP vision encoder in ComfyUI/models/clip, and the cgrkzexw-599808 adapter folder in ComfyUI/models/Joy_caption - all downloadable from HuggingFace directly or via the author's bundled Baidu links.
Same failure modes as the loader node, since they share the same underlying stack: UnboundLocalError on clip_processor or text_model with all model files confirmed present means transformers needs updating to at least 4.43.2, and a protobuf descriptor error on top of that fix means downgrading protobuf to 3.20.3 or setting PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python. If you're setting up fresh and don't have a specific reason to want this generation's exact output style, JoyCaption Beta One is the newer, simpler-to-run option - this one's worth reaching for mainly if an existing workflow or dataset was already built around alpha-two's particular phrasing.
Inputs (14)
| Name | Type | Default | Description |
|---|---|---|---|
| image | IMAGE | — | |
| llm_model | COMBO | 2 options: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2, unsloth/Meta-Llama-3.1-8B-Instruct | |
| device | COMBO | 1 options: cuda | |
| dtype | COMBO | 2 options: nf4, bf16 | |
| vlm_lora | COMBO | 2 options: text_model, none | |
| caption_type | COMBO | 9 options: Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru tag list, Booru-like tag list, +3 | |
| caption_length | COMBO | 31 options: any, very short, short, medium-length, long, very long, +25 | |
| user_prompt | STRING | — | |
| max_new_tokens | INT | 3008–4096 | — |
| top_p | FLOAT | 0.900–1 | — |
| temperature | FLOAT | 0.600–1 | — |
| cache_model | BOOLEAN | false | — |
| use_global_model | BOOLEAN | false | — |
| extra_optionsopt | JoyCaption2ExtraOption | — |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| text | STRING | — |