CLIPtion Beam Search (Integrated)

Want a caption for an image without loading a multi-gigabyte vision-language model into VRAM? That's the whole pitch here. CLIPtion Beam Search (Integrated) is a single node that turns the CLIP-L model you almost certainly already have loaded into an image captioner, using a ~100 MB decoder on top. It's a streamlined fork of pharmapsychotic/comfy-cliption, the December 2024 release that the community found "super extremely fast" - often beating Florence-2 and holding its own against JoyCaption on simple, precise subjects, at a fraction of the weight.

Here's where it sits in the captioning lineup. JoyCaption is the natural-language workhorse, WD14 is the Danbooru tagger for anime bases, and CLIPtion is the lightweight CLIP-native path: short, terse captions that are exactly as descriptive as CLIP's own vocabulary, ranked by how well they match the image in CLIP space. It's ideal for seeding an img2img prompt or captioning a LoRA dataset of single-character shots. Where it genuinely falls over - and this is community-tested, not a guess - is NSFW. Even light NSFW confuses it, because CLIP was never trained to describe that.

How it works

The name is honest. The node encodes your image with the CLIP vision encoder, feeds the resulting 1024-dim features through a small six-block transformer decoder (768 hidden, max 77 tokens) that does beam search over CLIP's own token vocabulary, then scores every finished beam by CLIP similarity between the candidate text and the image embedding and keeps the best one. That last rerank is the clever bit - the decoder proposes, CLIP arbitrates. One output per input image, and it handles a whole batch in one pass. A punctuation-spacing fixer cleans up the tokenizer's "cats , dogs ." artifacts on the way out.

The decoder loads on demand at caption time, not at workflow load, and stays cached between runs until you tell it otherwise.

The inputs that matter

• clip - a CLIP text encoder, and it must include CLIP-L or the node throws. Your SD1.5 or SDXL checkpoint counts.
• clip_vision - must be the CLIP-L (ViT-L/14) vision encoder; its 1024-dim features are a hard requirement.
• image - input image or batch.
• beam_width (default 9) - beams kept during search. Bump it for more candidates to rerank, trim it for speed. Range 1–64.
• unload_after_run (default on) - drops the decoder from VRAM after captioning. Great for tight VRAM, but flip it off when captioning a big batch so the decoder stays hot.
• force_cpu (optional) - run the decoder on CPU. It's small enough that this is actually usable.

Installing it

ComfyUI Manager works - search "Cliption Integrated" - or:

cd ComfyUI/custom_nodes
git clone https://github.com/easygoing0114/ComfyUI-cliption-integrated.git

Restart ComfyUI and it appears in the node menu under pharmapsychotic (a quirk of the fork: it keeps the original's category). First run downloads CLIPtion_20241219_fp16.safetensors (~117 MB) from Hugging Face into ComfyUI/models/cliption, so you need internet once. The README's recommended pairing is zer0int/CLIP-SAE-ViT-L-14, a high-accuracy CLIP-L fine-tune - drop the ~1.7 GB FP32 file into both ComfyUI/models/text_encoders and ComfyUI/models/clip_vision and load it twice. That's optional; a plain SDXL checkpoint with its built-in CLIP works too.

Where people get burned

• Wrong encoders. "Must use model which includes CLIP-L" or "Expected image features to have 1024 dimensions" means your clip or clip_vision isn't a ViT-L/14. That's the #1 failure.
• VRAM. CLIP text + vision eat ~0.5–1 GB on their own; on a tight card leave unload_after_run on, or force CPU.
• Expecting JoyCaption prose. You'll get short CLIP-flavored sentences, not paragraphs. That's the design, not a bug.