Simple LLM Caption

The name undersells it

SimpleLLMCaption is the workhorse of the Joy Caption Two PixelaiLabs pack, and "simple" doesn't mean weak. Wire in an image and it hands back a clean natural-language caption powered by Joy Caption Alpha Two - the captioner that's still the community's favorite for training data, with well over a hundred mentions in the first half of 2026 alone. No API, no key, no cloud. Everything runs on your GPU, and the whole stack downloads itself the first time you hit Run.

You reach for this when you're building a dataset: LoRA or fine-tune training, organizing an image folder, or just getting a solid description of a picture you can't place. It's the node the pack's Loader was built to feed.

How it actually works

Here's the trick that makes Joy Caption good: the image never goes through a generic vision-to-text model. Your image is resized to 384x384 and pushed through Google's SigLIP vision model, and the Joy Caption image adapter (an 86MB projection) converts those features into the Llama embedding space - 1152 dimensions in, 4096 out. Those image embeddings get spliced directly into a Llama 3.1 chat template where the image token would go, and generation runs. The 671MB LoRA adapter is what keeps the output clean.

That LoRA is the difference between "a woman standing in front of a building" and "Here is a caption: a woman standing in front of a building". Older Joy Caption implementations had to strip that chatter with post-processing; this pack bakes the fix into the weights, so captions come out ready to use.

Two behaviors worth knowing. First, the node forces re-execution whenever any input changes (it hashes its parameters to dodge ComfyUI's node cache), so captions refresh live as you tweak. Second, after every caption the models unload back to CPU. That frees up to ~10GB of VRAM between calls and keeps long sessions alive on 8GB cards - at the cost of a reload between generations, which is why each caption isn't instant.

The inputs that actually matter

The required trio is pipeline (from the Loader), image, and the two knobs that decide what you get:

• caption_type - Descriptive, Descriptive (Informal), Training Prompt, MidJourney, Booru Tags, Art Critic, or Social Media. Each maps to a different Joy Caption prompt template; Training Prompt and Booru Tags are where you'll live for dataset work.
• caption_length - any, very short, short, medium-length, long, very long. Note the hyphen: this node's enum says "medium-length", not "medium".

Everything else is optional and only fires if you set it. lora_trigger is the one worth knowing - put your rare trigger token in here and it gets prepended to every caption (ch9ractername, a woman ...), so training captions are born with the trigger attached. gender_age_replacement, hair_replacement, and body_size_replacement do regex swaps (put "black" in hair_replacement and "blonde hair" becomes "black hair"), and remove_tattoos / remove_jewelry strip those terms out entirely.

The output is a single STRING. Wire it to a Save Text node, or straight into a prompt input if you're going caption-to-image.

Install and first run

Standard custom-node fare:

cd ComfyUI/custom_nodes
git clone https://github.com/Pixelailabs/Joy_Caption_Two_PixelaiLabs.git
cd Joy_Caption_Two_PixelaiLabs
pip install -r requirements.txt

Then restart ComfyUI. ComfyUI Manager works too - search "Joy Caption Two" and install from there. Just know the first Run isn't free: it pulls SigLIP (~1.5GB), the Joy Caption adapters (~2.5GB), and the LLM you picked (~4-5GB). That's 6-8GB, one time. The console prints progress, and if a download dies it prints the HuggingFace links so you can grab the files by hand.

Where people get burned

• The first caption looks hung while models download. Give it a minute and watch the console.
• 8GB VRAM is the floor with the bundled 4-bit LLMs; 12GB is comfortable. An OOM on the first caption is almost always the reload step, not the node.
• bitsandbytes is a hard dependency for the 4-bit models. Manual installs that skip pip install -r requirements.txt fail at model load with a bitsandbytes error.
• If you symlink model folders out to a NAS: this pack writes to ComfyUI/models/Joy_caption, which won't follow your extra_model_paths. Leave it local.