LayerUtility: Load VQA Model

VQA stands for Visual Question Answering - not "describe this image" (that's captioning), but literally "answer this specific question about this image": is the person wearing glasses?, what color is the car?, is there text visible? This node loads the BLIP model that does that, and by itself it doesn't do much - it's a loader, meant to feed a companion question-answering node elsewhere in the pack that actually takes your question and the loaded model and produces an answer.

How it works

BLIP (Bootstrapping Language-Image Pre-training) is Salesforce's vision-language model line, and its VQA variant is specifically fine-tuned to take an image plus a text question and output a short text answer. This node just loads the weights onto your chosen device and hands them downstream as a VQA_MODEL object - it's the "load checkpoint" step, structurally identical in spirit to loading a diffusion checkpoint, just for a different kind of model.

Worth knowing going in: BLIP is an older model line (2022), and the community's general take on it - mostly in the captioning context, where BLIP has a well-earned reputation for producing formulaic, often inaccurate descriptions compared to newer captioners like Florence 2 or JoyCaption - is not glowing. VQA is a narrower, more constrained task than open-ended captioning (answering one specific question rather than freely describing a scene), so it holds up better there, but it's worth calibrating your expectations: don't expect GPT-4V-level reasoning out of a targeted yes/no or short-answer question.

The inputs and outputs that matter

• model - blip-vqa-base or blip-vqa-capfilt-large. The base model is smaller and faster; capfilt-large is trained with a larger, filtered caption dataset and generally answers a bit more reliably at the cost of size and speed.
• precision - fp16 or fp32. fp16 halves memory and is the default choice unless you're hitting precision-related quality issues (rare for this kind of model).
• device - cuda or cpu. CPU works but is meaningfully slower; fine for occasional single-image queries, painful in a batch loop.

Output: vqa_model, a VQA_MODEL object to wire into whatever question-answering node consumes it downstream.

How to install it

Search ComfyUI Layer Style in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/chflame163/ComfyUI_LayerStyle
pip install -r ComfyUI_LayerStyle/requirements.txt

Restart; it's under 😺dzNodes → LayerUtility. Unlike most of the pack's utility nodes, this one does pull down real model weights the first time you run it - BLIP's VQA checkpoints are auto-downloaded from Hugging Face on first use, so expect a pause (and some disk usage) the very first time this node executes, not on every run after.

Common issues

If the first run hangs or seems stuck, check your console - it's very likely mid-download of the model weights rather than actually frozen; give it time on a slow connection before assuming something's broken. If you're running this alongside a lot of other model-loading nodes and hitting out-of-memory errors, try precision: fp16 and confirm device is actually set to cuda (leaving it on cpu by accident won't crash, but it will be dramatically slower, which people sometimes mistake for a hang). And remember this node alone produces no answers - if nothing downstream is actually asking a question of the loaded model, you won't see any VQA output at all; check that you've got the matching question node wired in after this one.