TTP_JoyCaption2_Full

If you're training a LoRA, you already know the pain: captions are the difference between a model that learns what you meant and one that bakes in the wrong things. And if you've been burned by BLIP, you know why people don't trust auto-taggers with a dataset. This node is the fix. JoyCaption2 (shown in the node picker as TTP_JoyCaption2_Full) runs JoyCaption Alpha Two - the uncensored vision-language model the training community actually trusts - entirely inside ComfyUI. The name is a lie in the best way: it calls no API and needs no key. Everything happens on your GPU.

What it actually is

JoyCaption is fpgaminer's open, uncensored captioning VLM, built specifically because "describe it like a chatbot would" models kept dodging half the content in a training set. The Alpha Two stack is three moving parts: a SigLIP vision encoder, a small image adapter, and a Llama 3.1 8B LLM dressed up with a LoRA. This node wires those together for you, which is genuinely useful because the reference implementation is a pain to assemble by hand.

The model plumbing is the interesting bit. The LLM (unsloth/Meta-Llama-3.1-8B-Instruct by default, or the Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 variant) downloads itself on first run into ComfyUI/models/LLM/, and SigLIP does the same into clip_vision/. The one thing you place by hand is the Alpha Two checkpoint - clip_model.pt, image_adapter.pt, and the text_model LoRA - at ComfyUI/models/Joy_caption/cgrkzexw-599808, pulled from the fancyfeast/joy-caption-alpha-two space. The README's model table spells out the paths; get those right and the node is plug-and-play.

The inputs that matter

You don't need to touch all ~30 fields. The ones you'll actually set:

• image - feed it a batch and you get one caption per image back.
• caption_type - 9 modes, from plain "Descriptive" through "Booru tag list", "MidJourney", "Training Prompt", "Art Critic", even "Product Listing". Training Prompt is the one most people reach for.
• caption_length - words or a hard number ("any", "short", or a count up to 255). Longer isn't always better; word-count caps make the LLM actually tighten up.
• dtype - nf4 (≈10GB VRAM) or bf16 (≈19GB+). If you're under a 3090/4090, nf4 is the sane choice; the README recommends it for anything under 19GB.
• cache_model - keep the model loaded between runs instead of reloading the 8B every queue. Defaults off; flip it on for batch work.
• user_prompt - extra instructions appended to the caption request, handy for steering style.

Below those sit 17 extra-option toggles (lighting details, camera angle, watermark presence, SFW/NSFW classification, and so on) plus character_name - if you check "replace_character_names", the caption will refer to your character by name. The output is a single text list: one caption string per input image, ready to pipe into your training script or a text file writer.

Installing it

Easiest path: ComfyUI Manager, search Comfyui_JC2, install, restart. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/TTPlanetPig/Comfyui_JC2

Then install deps into your ComfyUI Python: accelerate, bitsandbytes, peft==0.12.0, transformers, sentencepiece, and triton-windows on Windows (pip install -r requirements.txt). Add the Alpha Two checkpoint, restart, and the first run downloads the LLM - give it a few minutes.

Where people get burned

• Bitsandbytes/quantization errors. "Using bitsandbytes 8-bit quantization requires the latest version" is a classic; pip install -U bitsandbytes fixes most of it.
• VRAM OOM at bf16. If you're not on a 24GB card, use nf4. The node even tries to clean VRAM before loading when it detects you're tight.
• First-run downloads. A ~16GB Llama is not instant. Don't panic at the silent console.
• Macs. The device list only offers cuda:0 - this is a CUDA node. Apple Silicon users have patched JC2.py to use mps, but it's not supported out of the box.

If you don't want all 17 toggles cluttering the node, grab the JoyCaption2_simple sibling instead and wire it to the pack's ExtraOptionsNode.