Nodes/JoyCaptionAlpha Two for ComfyUI/Joy Caption Two Advanced
ComfyUI Node

Joy Caption Two Advanced

Full control over JoyCaption's prompt — custom text, sampling, and the 17 extra instructions

By EvilBT·Created 2 years ago·Updated about a year ago· 721
Joy Caption Two Advanced
  • joy_two_pipeline
  • image
  • extra_options
  • STRING
caption_type
caption_lengthlong
name
custom_prompt
low_vramfalse
top_p0.90
temperature0.60

This is Joy_caption_two with every knob exposed: an extra_options socket for fine-grained content control, a name field, a custom_prompt that can override the auto-built prompt entirely, and sampling controls (top_p, temperature) the plain node doesn't give you. If the basic node's default captions are close but not quite what your training set needs - too generic, missing camera detail, not explicit enough about NSFW content, whatever - this is where you fix it.

The reason this matters more than it sounds: for LLM-encoder models like Flux, caption quality is the highest-leverage thing you control after picking your dataset. Get the phrasing wrong - describing things you wanted to stay fixed, or leaving out things you wanted the model to vary - and you'll fight the LoRA for it later. This node is where you get that phrasing right at the source instead of hand-editing caption files afterward.

How it works

Same pipeline as the plain node - SigLIP reads the image, Llama writes the caption - but the prompt sent to the LLM is now assembled from more pieces: your caption_type, your caption_length, whatever's wired into extra_options, and your name if you're using it. Or you skip all of that and hand it a custom_prompt directly, in which case the node just runs your prompt verbatim instead of building one.

The inputs and outputs that matter

  • joy_two_pipeline, image, caption_type, caption_length, low_vram - same as the plain node.
  • extra_options - plug in the output of a Joy_extra_options node here to layer on any of its 17 content toggles (lighting, camera detail, PG-only, sfw/nsfw labeling, and so on).
  • name - a plain string. Only matters if you've turned on the extra option that says "refer to them as {name}" - without that toggle, this field does nothing.
  • custom_prompt - write your own instruction instead of relying on caption_type/caption_length/extra_options. This fully replaces the built prompt when set, so it's an escape hatch when the preset system doesn't get you where you want.
  • top_p (default 0.9) and temperature (default 0.6) - standard LLM sampling controls. Lower temperature for more consistent, repeatable captions across a batch; raise it if captions feel stiff or repetitive. Most people never need to touch these - they were added later in the pack's life specifically for people who wanted more variety.

Output is a single STRING - the caption. Wire it into a text-preview/save node, same as the plain version.

How to install it

Through ComfyUI Manager: search "JoyCaptionAlpha Two for ComfyUI" and install. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/EvilBT/ComfyUI_SLK_joy_caption_two.git
pip install -r ComfyUI_SLK_joy_caption_two/requirements.txt

Restart ComfyUI. Same model dependency chain as every node in this pack: Joy_caption_two_load upstream handles SigLIP and your chosen Llama checkpoint (both auto-download), but the JoyCaption adapter itself has to be manually pulled from the fancyfeast/joy-caption-alpha-two HuggingFace space into models/Joy_caption_two - that step doesn't happen automatically.

Common issues & troubleshooting

custom_prompt doesn't seem to be doing anything, or output looks like the default caption anyway. Make sure it's actually non-empty - an empty string falls back to the built-in prompt from caption_type/caption_length. If you want your custom prompt to be the whole instruction, don't also expect extra_options toggles to apply on top of it; when you write your own prompt, you own all of it.

Output is inconsistent run to run on the same image. That's temperature doing its job - it's a sampling model, not a deterministic function. Drop temperature toward 0 if you need repeatability, e.g. for regression-testing a workflow.

VRAM issues, empty output, or load failures. These almost always trace back to Joy_caption_two_load - check the adapter files are actually in models/Joy_caption_two, and flip low_vram on here if you're tight on memory. The pack was built and tested on 8GB VRAM using the 4-bit Llama checkpoint, so that combination is your safest bet if you're resource-constrained.

Need to do this across a whole folder instead of one image at a time. That's Batch_joy_caption_two_advanced - identical option set, plus batch-specific fields for input/output directories and file naming.

CategorySLK/LLM

Inputs (10)

NameTypeDefaultDescription
joy_two_pipelineJoyTwoPipeline
imageIMAGE
extra_optionsExtra_Options
caption_typeCOMBO9 options: Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru tag list, Booru-like tag list, +3
caption_lengthCOMBOlong31 options: any, very short, short, medium-length, long, very long, +25
nameSTRING
custom_promptSTRING
low_vramBOOLEANfalse
top_pFLOAT0.900–1
temperatureFLOAT0.600–1

Outputs (1)

NameTypeDescription
STRINGSTRING