Joy_caption_alpha_run

Joy_caption_alpha_run is the single-image inference node for JoyCaption Alpha Two - the model that made caption quality a solved problem for most dataset work. Pair it with Joy_caption_alpha_load upstream and, optionally, Joy_caption_alpha_prompt to build the exact instruction you want, and you get natural-language captions that actually describe what's in the frame. On a 4090 the pack's own README quotes roughly 4–5 seconds per image for batch marking, so single images are near-instant once the model is loaded.

How it works

Same architecture as the pre-alpha: the image is resized to 384×384, normalized, run through the (custom) SigLIP encoder, and the image adapter converts the features into Llama embedding space. The prompt is wrapped in a chat template (You are a helpful image captioner. system message), the image tokens get inserted at the right spot, and the LLM generates the caption with default sampling (temperature 0.6 / top-p 0.9 per the source).

Two parameters are worth actually understanding, because they're this pack's answer to the "my caption node ate my workflow's VRAM" problem:

• cache - False (default) unloads the entire caption pipeline after every run: it calls clearCache(), empties the CUDA cache, and runs garbage collection. Perfect for a one-off caption inside a busy workflow. True keeps the pipeline resident for fast repeats but leaves ~10 GB parked.
• low_vram - when True, the node calls unload_all_models() before captioning, evicting whatever diffusion models are loaded so the caption pipeline fits. Flip this when the captioner is OOMing next to your checkpoint.

There's also a seed input, but note the source passes it along without actually forcing deterministic sampling through it - treat it as decoration.

Inputs and output

• Inputs: JoyPipeline_alpha (from the loader), image (IMAGE), prompt (multiline - wire in a Joy_caption_alpha_prompt output here for the fancy modes), max_new_tokens (10–4096, default 1024), temperature (0–1, default 0.7), cache, low_vram, seed.
• Output: one STRING - the caption.

That string is your dataset caption, your image-to-prompt input, whatever. Feed it to a text-save node, into a prompt encoder, or paste it into your training config.

Installing

Standard pack install - ComfyUI Manager (search "Comfyui_CXH_joy_caption") or clone and pip:

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

Then restart, and make sure the Alpha Two checkpoint is at models/Joy_caption_alpha/ (see the Joy_caption_alpha_load article - that's where the manual download step lives).

Troubleshooting

• OOM when captioning inside a generate workflow: that's what low_vram is for. Turn it on.
• Slow repeat captions: flip cache on, at the cost of VRAM.
• Only NVIDIA: the code calls .to('cuda') directly, so this is a CUDA-only node.
• Weird captions: Alpha Two occasionally loops or picks the wrong mode - it's a known model quirk, not your workflow. Re-run with a different seed or shorten max_new_tokens. If you want the controlled modes, don't hand-write the prompt; build it with Joy_caption_alpha_prompt.