Joy Caption Two Load

Every other node in this pack needs a loaded model to actually caption anything, and this is the node that does the loading. Joy_caption_two_load pulls in a vision encoder plus an 8B Llama, wires them together into JoyCaption's captioning pipeline, and hands you a single pipeline object to plug into every downstream node. You only need one of these per workflow - load once, caption as many images as you want off the same pipeline.

If you haven't run into JoyCaption before: it's the free, open, uncensored VLM that's become the go-to for captioning LoRA training sets, especially anything that needs to cover NSFW content evenly rather than refusing or hand-waving around it. It's genuinely the community favorite for this - the alternative is Florence-2 (also decent, what Fluxgym defaults to) or WD14 tagging if you're training on a Danbooru-tagged base like Illustrious. BLIP, if you're wondering, is the one everyone tells you to stop using; it's old and produces garbage captions.

How it works

Under the hood JoyCaption is a LLaVA-style setup: a SigLIP vision encoder reads the image, and a fine-tuned Llama-3.1-8B does the actual writing, conditioned on what SigLIP saw. This node assembles that pair (plus a small adapter that bridges the two) into one object and loads it onto your GPU.

The input that matters

There's exactly one required input, model - a dropdown of four Llama checkpoints:

• unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit - 4-bit quantized. This is what the pack author actually runs (on 8GB VRAM), and it's the sane default unless you have VRAM to spare.
• unsloth/Meta-Llama-3.1-8B-Instruct - the full-precision version. Needs meaningfully more VRAM for not much of a quality jump on captioning.
• John6666/Llama-3.1-8B-Lexi-Uncensored-V2-nf4 and Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 - swapped-in "uncensored" Llama finetunes rather than the stock instruct model. The pack author tried the nf4 one and rated the results good; worth trying if the default feels like it's pulling punches on spicy content.

The output is a single JoyTwoPipeline - connect it to the joy_two_pipeline input on Joy_caption_two, Joy_caption_two_advanced, or either batch node.

How to install it

Through ComfyUI Manager: search "JoyCaptionAlpha Two for ComfyUI" and install. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/EvilBT/ComfyUI_SLK_joy_caption_two.git
pip install -r ComfyUI_SLK_joy_caption_two/requirements.txt

Then restart ComfyUI - but restarting isn't the end of setup, because this is the node that actually triggers the model downloads, and one of the three pieces doesn't auto-download.

The vision encoder (google/siglip-so400m-patch14-384) downloads automatically the first time you run this node, into models/clip/siglip-so400m-patch14-384. The Llama checkpoint you picked also auto-downloads, into models/LLM/<model-name>. The JoyCaption adapter itself does not. You have to manually grab the cgrkzexw-599808 folder from the fancyfeast/joy-caption-alpha-two HuggingFace space and drop its contents into models/Joy_caption_two. Skip this step and the loader will fail - this is the single most common reason people can't get this node working at all.

Common issues & troubleshooting

It loads, but the caption comes out empty or the load errors out. Check models/Joy_caption_two actually has the adapter files in it. Auto-download only covers two of the three pieces.

You're on 6-8GB VRAM and it's slow, crashing, or won't fit. Use the bnb-4bit model - it's what the pack was built and tested against. Even then, budget disk space generously: SigLIP plus an 8B Llama plus the adapter is comfortably over 10GB pulled down between models/ and your HuggingFace cache. If you're still tight, every downstream node also has a low_vram toggle worth flipping on.

Worth knowing going in: this node wraps JoyCaption "Alpha Two" specifically - you can see it in the caption-type list downstream, which matches Alpha Two's original nine modes exactly. The upstream JoyCaption project has since moved on to a newer "Beta One" release with better stability and a couple of new modes, but this ComfyUI pack hasn't been updated to match (last release was October 2024). That's not a dealbreaker - Alpha Two still captions perfectly well - just don't expect Beta One's improvements here.