MiniCPM-V-4 (GGUF) Vision Infer

This is the node that finally makes the vision LLM do something. You feed it an image, you ask a question, and you get back a real sentence - not tags, not a score, actual language about what's in the picture. That's the whole selling point of the MiniCPM-V-4 pack, and why people put it in their ComfyUI at all: local, open, no API key, and it can caption your generations well enough to feed back into your own prompts.

It's the natural second half of the loader. The loader hands you a MINICPM_HANDLE; this node takes that handle, an image, and a text prompt, and returns a STRING you can save, display, or pipe anywhere a string flows.

How it works

Mechanically it's a chat-completion call over llama.cpp. The node converts your ComfyUI image tensor into a PNG, base64-encodes it into a data URI, and sends it to the model as an image_url message alongside your prompt. It also does a few things you'd otherwise never think about: it hashes the image and appends a timestamp to the prompt to bust llama.cpp's KV-cache, so it doesn't "remember" the previous picture and answer with stale text. When force_new_context is on (the default), it actively resets the model and clears the KV cache between runs - the right choice for ComfyUI, where you're usually judging one image at a time.

Then it cleans the output - stripping chat tags, role prefixes, and stray markers - and returns the text.

The inputs that matter

Three required, four you'll touch occasionally:

• handle - the MINICPM_HANDLE from the loader. Forget to wire it and you get "Invalid handle - model not loaded."
• image - any IMAGE. One image, unless you read below.
• prompt - default "Describe this image in detail." This is where you actually get value. "List the visible text." "Describe the lighting and composition." "Write a caption that would prompt this back."
• max_tokens (512, capped 64–2048), temperature (0.7), top_p (0.9), top_k (40), repeat_penalty (1.1) - the usual sampler knobs. Lower temperature for captioning, higher for more creative descriptions.

The output is a single text string. Wire it to a Save Text node, a Display Any, or back into a workflow that builds prompts from descriptions.

The honest gotchas

The README promises multi-image input - "send an image batch, reference them in the prompt." Read the code and the truth is: it only uses the first image in the batch. _pil_from_comfy grabs x[0] and moves on. So that "compare these two images" workflow won't work no matter how you prompt it. Plan around it, or patch the node.

Second, if the model returns fewer than 10 characters of content, the node decides something went wrong and returns a huge troubleshooting error message as the text output. That's by design (the author's own guidance baked in: check the mmproj, try Q8, update llama-cpp-python). But it means your Save Text node can end up holding a wall of debugging text instead of a caption. If the output looks like an error, it isn't the model being coy - read it, it's telling you what to fix.

One more: the README describes a separate "Vision Prompt Builder" node for chat tags and <image> tokens. It's not actually registered by the pack - you won't find it in your node list. It doesn't matter. This node always passes your image as an image_url message and your prompt as separate text, so there's no <image> token to worry about. Just type your prompt and go.

If the image seems to be ignored

That's the classic VLM failure and it's worth a checklist: make sure llama-cpp-python is actually CUDA-enabled (a CPU-only wheel will still load, just slowly), confirm the loader printed a successful load to the console, and bump max_tokens if you're getting cut-off answers. Images ignored is almost always the loader, not this node - the infer node is the one doing exactly what it's told.