Blip Processor Node

You've got an image sitting in a workflow and you want to ask it something - "how many people are in this shot?", "is the sky clear?", "is there a red car?" - and maybe feed the answer into a prompt or a branch. That's exactly what the Blip Processor Node is for. It's the headliner of the small muhammederem/blip-comfyui pack and about the simplest way to do visual question answering (VQA) inside ComfyUI: no cloud service, no API key, just a model that auto-downloads on first run.

Don't confuse this with BLIP captioning. If you've spent any time in the training community you've heard "never use BLIP for captions," and that's fair - for LoRA training data it's been dead for years, replaced by WD14 taggers and newer captioners like JoyCaption or Florence-2. This is a different animal: it's question answering against a single image, and for that job the venerable Salesforce/blip-vqa-base still holds up fine. The honest caveat is that the model dates to 2022 and was trained mostly on COCO-style photographs. Ask it "how many cars?" or "is there a blue car?" and it's solid. Ask it to critique your anime render and it will confidently tell you things that are, let's say, optimistic.

How it works

The node loads Salesforce/blip-vqa-base through the transformers library, once, using a singleton, and hard-pins it to the GPU. Each execution takes your image tensor, grabs the first frame if you fed it a batch, converts it to a PIL image, then for every question encodes image + text, runs the model's generate loop, and decodes a short answer like "a cat sitting on a chair."

Two things to know about that load. It happens on the first execution, not at startup, so the first run will sit there for a while - that's the roughly 1GB model downloading into your HuggingFace cache (~/.cache/huggingface), not your ComfyUI/models folder. This is a classic point of confusion: people go hunting for a checkpoint that was never going to be there.

Inputs and outputs

The inputs are refreshingly plain. image is the IMAGE tensor straight out of LoadImage. question is required and defaults to "What is in the image?". Then there are nine optional slots, question_1 through question_9 - leave them blank and they're silently skipped, so your practical cap is ten questions per image. Output is a single LIST_STRING: the answers, in the same order you asked the questions.

Installing it

The usual two steps. Via ComfyUI Manager, search "blip-comfyui" (or "BLIP Vision-Language Model Integration"). Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/muhammederem/blip-comfyui

Then restart ComfyUI. Watch the dependency pins: the pack locks transformers==4.44.0, torchvision==0.16.2, and Pillow==11.0.0 with exact == versions. If you run a lot of other nodes, one of those can drag a shared library backward and break something unrelated. It's the classic "I installed one node and my other nodes started erroring" scenario, so if things go weird after install, check those pins first.

Where people get burned

• It's CUDA-only, full stop. The code hardcodes .to("cuda") with no CPU fallback. No Nvidia GPU, no dice - this is why the README stresses CUDA setup.
• Answers aren't visible until you display them. A bare LIST_STRING output renders as nothing readable. Wire it into the pack's other node, List to Text, then a ShowText node (from pysssss's ComfyUI-Custom-Scripts) - the pack's own example workflow is exactly that chain.
• Ask questions the model can actually answer. Specific, countable, yes/no questions work great; open-ended "describe this in detail" is what captioners are for.
• Only the first image in a batch gets processed. Feed it a list of images and it looks at frame zero.

That's the whole thing. It won't replace a modern VLM, but as a "does this image have X?" gate or a quick sanity check mid-workflow it's a satisfying little hammer - and it costs you nothing but a gigabyte of disk.