Replicate salesforce/blip

Feed it an image and it tells you what's in it. BLIP is Salesforce's image-understanding model, and it does three jobs: caption an image, answer a question about an image, or check whether a caption matches an image. The most common use is the first one - quick, plain-English captions, the kind you'd want for auto-labeling a dataset or dropping a description into a prompt chain. It's an old, small, dependable workhorse rather than anything cutting-edge.

Like everything in this pack it's a Replicate node, so it runs in the cloud: your image goes up to Replicate, the model runs there, the text comes back. You need a Replicate account and a REPLICATE_API_TOKEN, you pay per run, and nothing installs locally beyond the wrapper.

How it works

BLIP was trained to bridge vision and language, so it can both describe an image and answer questions grounded in it. In captioning mode it generates a short sentence; in VQA mode it reads your question and answers from the image; in matching mode it scores how well a caption fits. It's a single model wearing three hats, and the task input picks which.

The inputs and outputs that matter

• image (required) - the picture to analyze.
• task - image_captioning (describe it), visual_question_answering (answer a question about it), or image_text_matching (score a caption against it).
• question - the question to answer, used with visual_question_answering.
• caption - the caption to check, used with image_text_matching.

Output is a single STRING - the caption, the answer, or the match result - which you wire into a show-text node, a CLIP/text encode, or onward to an LLM node. force_rerun bypasses Replicate's cache when identical inputs would return the same string.

How to install it

One install covers every Replicate node in the pack.

• ComfyUI Manager: search ComfyUI-Replicate, install, restart.
• Manual:

  cd ComfyUI/custom_nodes
  git clone https://github.com/replicate/comfyui-replicate
  cd comfyui-replicate
  pip install -r requirements.txt
  

  then restart.

Nothing downloads to your disk - just the replicate pip package. Set your token first: export REPLICATE_API_TOKEN="r8_..." (replicate.com/account/api-tokens).

Common issues & troubleshooting

The question mode isn't working. VQA needs the question field filled and task set to visual_question_answering. Same for matching: fill caption and set the task. Leaving task on captioning ignores those fields entirely.

The caption is short and generic. That's BLIP being BLIP. It's an older, lightweight captioner - it reliably gets the gist ("a dog running on a beach") but won't give you a rich, detailed paragraph. That's fine for quick labels or a rough prompt seed. If you want detailed, promptable descriptions, run a CLIP-interrogator (for artist/style tags) or a modern vision-language model instead.

It missed or misread something in a busy image. Simple captioners get tunnel vision on the dominant subject. If you need it to attend to a specific detail, switch to VQA and ask - "what color is the car?" - rather than hoping the caption mentions it.

Same output on a re-run. Replicate returned the cached result for identical inputs. Set force_rerun if you need it recomputed.