MiniCPM VQA Simple

This node runs a real, fairly capable vision-language model directly in your ComfyUI graph: MiniCPM-V 2.6, specifically its int4-quantized build, so you feed it an image and a question in plain text and get a text answer back - no external API call, no key to manage. Think of it as a captioning node with a brain: instead of a fixed "describe this image" template, you write whatever question you actually want answered, and it answers that.

MiniCPM-V's presence in the ComfyUI world isn't a Mixlab-only thing - other packs wrap the same family of models for similar visual-QA and auto-captioning workflows, and it's a reasonably popular corner of the local-VLM space precisely because most alternatives either need a cloud API or a lot more VRAM. The int4 quantization the README specifically calls out is what makes this version practical to run locally at all: it needs roughly 7GB of VRAM, which is real but not out of reach for a mid-range card, whereas an unquantized multimodal model of similar size routinely wants far more.

The inputs map onto that VQA framing directly. images is what you're asking about. text is your question or instruction, a multiline STRING - the more specific you are here, the more useful the answer; a vague prompt gets a vague answer, same as with any LLM. seed (default -1) and temperature (default 0.7) control how deterministic versus varied the phrasing comes back - lower temperature for consistent, literal answers, higher if you want more natural variation across repeated runs. extract_keywords is a boolean toggle: turn it on and the node also tries to pull a condensed keyword list out of its own answer, which lands in the separate keywords output - useful if you want short tags for a caption database rather than a full sentence. keep_model_loaded decides whether the model stays resident in VRAM between runs or gets released after each execution; leave it on if you're running this node repeatedly in the same session and want to skip the reload cost each time, but remember that means it's holding onto roughly 7GB the whole time it's on, even between unrelated jobs.

Two outputs: result, the model's full text answer, and keywords, the condensed extraction if you enabled it (otherwise it'll typically come back empty or unused).

Model setup is a real step here, not optional. Per the README, you need the MiniCPM-V-2_6-int4 weights from openbmb/MiniCPM-V-2_6-int4 on Hugging Face - this isn't bundled with the pack install, it's a separate download the node expects to find before it'll run. Installing the pack itself is the usual path:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
cd comfyui-mixlab-nodes
install.bat

or through ComfyUI Manager (search "comfyui-mixlab-nodes"), or pip3 install -r requirements.txt in a venv. Restart ComfyUI, then fetch the model weights before your first run - a fresh install with no model downloaded is the most common reason this node fails immediately, and it looks like a broken node when it's really just a missing file. If you're tight on VRAM even at int4, keep_model_loaded is the first setting to turn off, and dropping it should free that memory back up between runs.