Nodes/ComfyUI_LayerStyle_Advance/LayerUtility: JoyCaption2(Advance)
ComfyUI Node Runs on cloud

LayerUtility: JoyCaption2(Advance)

JoyCaption2 — the original JoyCaption-alpha-two captioner, wired for ComfyUI

By chflame163·Created 2 years ago·Updated 4 months ago· 696
LayerUtility: JoyCaption2(Advance)
  • image
  • extra_options
  • text
llm_model
device
dtype
vlm_lora
caption_type
caption_length
user_prompt
max_new_tokens300
top_p0.90
temperature0.60
cache_modelfalse
use_global_modelfalse

This is JoyCaption's earlier "alpha-two" generation doing actual captioning work - everything the loader node (Load JoyCaption2 Model) sets up gets used here. fpgaminer built the whole JoyCaption line to solve one specific problem: every good vision-language model at the time was either expensive and censored (ChatGPT) or open but weak and still censored on anything past PG (CogVLM). JoyCaption trades some raw benchmark polish for being genuinely uncensored and free, which is exactly why it became the default recommendation for captioning LoRA training sets rather than something like BLIP, which the community wrote off years ago as producing formulaic, inaccurate captions.

Unlike the newer Beta One node, this one bundles the model config directly rather than taking a pre-loaded handle for everything - you still wire in llm_model, device, dtype, and vlm_lora here even with a separate loader existing, and there's a use_global_model toggle specifically for sharing a model already loaded elsewhere in your graph instead of loading a fresh copy. caption_type gives you nine formats: Descriptive, a looser Descriptive (Informal), Training Prompt (shaped specifically for feeding back into diffusion training, which is the whole reason JoyCaption's training-prompt mode exists as a distinct format from plain description), MidJourney-style, two Booru-style tag list formats, and three non-training formats (Art Critic, Product Listing, Social Media Post) for when you just want a description rather than dataset captions. caption_length controls verbosity from any/very short up through numeric targets. user_prompt, if filled in, overrides caption_type and any extra_options entirely - use it for full manual control. top_p, temperature, and max_new_tokens are standard sampling controls. cache_model keeps the model resident between runs rather than reloading it each execution, which is worth turning on if you're captioning a batch and don't want the load cost paid repeatedly. The optional extra_options input takes a JoyCaption2ExtraOption handle for fine-grained control over what gets included or excluded (lighting, camera angle, watermark mentions, and similar). Output is text.

Install the pack through ComfyUI Manager (search "ComfyUI Layer Style Advance") or git clone https://github.com/chflame163/ComfyUI_LayerStyle_Advance into custom_nodes/, restart, run the requirements installer. This node needs three separate model pieces in place: the LLM in ComfyUI/models/LLM, the SigLIP vision encoder in ComfyUI/models/clip, and the cgrkzexw-599808 adapter folder in ComfyUI/models/Joy_caption - all downloadable from HuggingFace directly or via the author's bundled Baidu links.

Same failure modes as the loader node, since they share the same underlying stack: UnboundLocalError on clip_processor or text_model with all model files confirmed present means transformers needs updating to at least 4.43.2, and a protobuf descriptor error on top of that fix means downgrading protobuf to 3.20.3 or setting PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python. If you're setting up fresh and don't have a specific reason to want this generation's exact output style, JoyCaption Beta One is the newer, simpler-to-run option - this one's worth reaching for mainly if an existing workflow or dataset was already built around alpha-two's particular phrasing.

Category😺dzNodes/LayerUtility

Inputs (14)

NameTypeDefaultDescription
imageIMAGE
llm_modelCOMBO2 options: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2, unsloth/Meta-Llama-3.1-8B-Instruct
deviceCOMBO1 options: cuda
dtypeCOMBO2 options: nf4, bf16
vlm_loraCOMBO2 options: text_model, none
caption_typeCOMBO9 options: Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru tag list, Booru-like tag list, +3
caption_lengthCOMBO31 options: any, very short, short, medium-length, long, very long, +25
user_promptSTRING
max_new_tokensINT3008–4096
top_pFLOAT0.900–1
temperatureFLOAT0.600–1
cache_modelBOOLEANfalse
use_global_modelBOOLEANfalse
extra_optionsoptJoyCaption2ExtraOption

Outputs (1)

NameTypeDescription
textSTRING