ComfyUI Node

Joy Caption Two

One image in, one natural-language caption out

By EvilBT·Created 2 years ago·Updated about a year ago· 721
Joy Caption Two
  • joy_two_pipeline
  • image
  • STRING
caption_type
caption_lengthlong
low_vramfalse

This is the plain version - feed it one image, pick a style and a length, get back a caption string. No custom prompt, no extra-options bank, no sampling knobs. If you're captioning a handful of images to sanity-check the model before committing to a full batch run, or you just want a quick caption for a single image without building out the advanced node's extra sockets, this is the one you want.

It exists in the pack because most of the time you don't need the advanced controls. JoyCaption's whole appeal for LoRA work is that it writes natural-language captions without refusing or dancing around NSFW content the way a lot of captioners do - that's the reason it's become the community default for Flux-style training sets over Florence-2 or hand-captioning, at least until your dataset is small enough that hand-captioning wins on precision.

How it works

It takes the JoyTwoPipeline from Joy_caption_two_load, runs your image through the SigLIP vision encoder, and has the Llama side write a caption in whichever style you asked for. Under the hood it's building the same kind of instruction prompt that Joy_caption_two_advanced lets you fully customize - this node just picks reasonable defaults and doesn't expose the prompt to you.

The inputs and outputs that matter

  • joy_two_pipeline - the pipeline from Joy_caption_two_load. Required, and there's no way around loading a pipeline first.
  • image - the image to caption.
  • caption_type - nine styles: Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru tag list, Booru-like tag list, Art Critic, Product Listing, Social Media Post. For LoRA training data, Descriptive or Training Prompt are the usual picks - Descriptive gives you full sentences, Training Prompt tries to mimic how a person actually writes an SD-style prompt (messier, tag-and-phrase mixtures).
  • caption_length - named lengths (any / very short / short / medium-length / long / very long, default long) or an exact word-count target in steps of ten if you want consistent-length captions across a dataset.
  • low_vram - off by default; flip it on if you're tight on VRAM. It trades some speed for a smaller footprint.

Output is a single STRING - the caption text. This node isn't an output node itself, so wire it into something that shows or saves text (a text-preview node, or straight into whatever's building your training file) or you'll never actually see what it wrote.

How to install it

Through ComfyUI Manager: search "JoyCaptionAlpha Two for ComfyUI" and install. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/EvilBT/ComfyUI_SLK_joy_caption_two.git
pip install -r ComfyUI_SLK_joy_caption_two/requirements.txt

Restart ComfyUI. This node itself needs nothing extra - all the model weight is on Joy_caption_two_load upstream: SigLIP and your chosen Llama checkpoint auto-download, but the JoyCaption adapter has to be pulled manually from the fancyfeast/joy-caption-alpha-two HuggingFace space into models/Joy_caption_two. Get that pipeline working first; this node is just the thin captioning call on top of it.

Common issues & troubleshooting

Caption comes back empty or the node errors. That's almost always an upstream problem with Joy_caption_two_load - usually the manually-downloaded adapter files missing from models/Joy_caption_two. Fix it there, not here.

Running out of VRAM mid-caption. Turn on low_vram. If you're still tight, the pack itself was built and tested on 8GB VRAM using the 4-bit Llama checkpoint - if you picked the full-precision model on the loader node, that's usually the fix.

One caption at a time is too slow for a real dataset. That's what Batch_joy_caption_two is for - same options, but it walks a whole folder instead of one image per graph run.

Captions feel generic or too tame for your use case. This node doesn't take extra_options or a custom_prompt - those live on Joy_caption_two_advanced. Step up to that node once you need finer control over content, tone, or NSFW handling.

CategorySLK/LLM

Inputs (5)

NameTypeDefaultDescription
joy_two_pipelineJoyTwoPipeline
imageIMAGE
caption_typeCOMBO9 options: Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru tag list, Booru-like tag list, +3
caption_lengthCOMBOlong31 options: any, very short, short, medium-length, long, very long, +25
low_vramBOOLEANfalse

Outputs (1)

NameTypeDescription
STRINGSTRING