CLIPTextEncodeBLIP

CLIPTextEncodeBLIP does one thing that sounds like magic until you try it: you feed it an image, it looks at it, writes a caption, splices that caption into your prompt, and hands you a CONDITIONING - exactly what a normal CLIPTextEncode would give you. Same wire into the sampler, zero extra steps. It's the "describe this image back to the model" trick, and for img2img work where you want the model to know what it's looking at, it's the most direct way to get there.

A fair warning before you fall in love: this node is a 2023 relic from the early ComfyUI days, and it shows. The pack's README announces that "BLIP is now officially integrated into CLIPTextEncode," which never really happened anywhere but this repo. It's unmaintained, gets essentially zero search traffic, and BLIP itself has been dead weight in the community for years - the consensus in the KB is that BLIP captions are formulaic and often wrong. That reputation is earned in the training-data world. For live inference it's a different job, and here the honest verdict from people who actually ran it: fine for a simple subject (a portrait, a single object), garbage on anything compositionally complex. Set expectations and it's still a great little tool.

How it works

Under the hood it's the real Salesforce BLIP decoder, vendored into the pack. On each run it resizes your image to 384×384, loads the model_base_capfilt_large.pth checkpoint (downloaded automatically on first run into ComfyUI/models/blip/ - a big download, over a gigabyte), and does a greedy beam-width-1 caption decode bounded by your min/max length settings. The caption then replaces the BLIP_TEXT placeholder in your text, and the whole thing is encoded with the advanced_encode routine from the AdvancedCLIPText playbook - that's why you get those two weird dropdowns.

The inputs that matter

Only a few are worth touching:

• image - what gets captioned. Wire in your Load Image (or an img2img source).
• clip - the CLIP model you're conditioning, same as CLIPTextEncode.
• min_length / max_length - caption length in tokens (defaults 5/20). Short captions read as "a photo of a woman"; bump max_length toward 30–40 for more detail.
• string_field - your prompt template. Default is {{BLIP_TEXT}}, and this is a trap.

That last one deserves its own paragraph, because the default quietly does the wrong thing. The code does string_field.replace("BLIP_TEXT", caption) - only the inner token gets replaced, so if you keep the {{BLIP_TEXT}} default your final prompt literally contains {{...}} braces around the caption, which the tokenizer happily encodes as noise. Change it to a plain BLIP_TEXT, as the README shows: "a photo of BLIP_TEXT, medium shot, intricate details".

token_normalization and weight_interpretation - leave them on none and comfy. Those are AdvancedCLIPText weighting options for fine-grained per-token emphasis; defaults behave like a regular CLIPTextEncode. Output is a single CONDITIONING that plugs into the positive (or negative, if you're feeling chaotic) input of your sampler.

Installing it

ComfyUI Manager can find it as comfy_clip_blip_node, or clone it:

cd ComfyUI/custom_nodes
git clone https://github.com/paulo-coronado/comfy_clip_blip_node

Then the one dependency ComfyUI doesn't already ship: fairscale.

# Windows portable: ComfyUI_windows_portable\python_embeded\python.exe -m pip install fairscale
pip install fairscale==0.4.4

Here's the catch: the pack's requirements.txt pins transformers==4.15.0 and timm==0.4.12 - ancient versions that will fight with modern ComfyUI (and the code itself sniffs pip for exactly transformers==4.26.1 to pick a code path). Do not run pip install -r requirements.txt; you'll downgrade your environment and break other nodes for a pack this old. Install fairscale alone and let everything else ride.

Where people get burned

The BLIP code imports from fairscale.nn.checkpoint.checkpoint_activations import checkpoint_wrapper, the single most fragile import in the ecosystem - it breaks on newer fairscale/PyTorch combos, so pin 0.4.4 if you get an ImportError. Two more quirks worth knowing: the node reloads the ~1.4GB model into VRAM on every queue run (slow first frame, and it'll hurt on a small card), and it calls os.chdir() as a side effect, which can confuse relative paths elsewhere in your workflow. A real user on r/comfyui tried exactly this node, hit an error, and the thread's best advice was the one that still stands: auto-captioning is reliable for simple subjects and quickly falls apart on busy scenes. Treat it as a prompt-engineering assist, not a caption oracle, and it'll earn its place.