MiniCPM-Plus: V-2.6

The node people actually reach for in this pack

MiniCPM_V_2_6 is the reason the ComfyUI-MiniCPM-Plus pack exists. It's a vision-language model - OpenBMB's MiniCPM-V-2.6, an 8B multimodal model - that looks at an image and writes natural-language descriptions. In ComfyUI terms that means one node does the job that used to take a WD14 tagger, a CLIP interrogator, and a lot of patience: it turns an image into a prompt.

That's the workflow the README leads with, and it's the one you'll actually use. Drop a reference image in, ask it to describe it as a prompt, and wire the result into a CLIP Text Encode. On an LLM-encoded model like Flux, a caption written by an LLM is exactly the input the encoder was trained to read - you're captioning with the same kind of model that wrote the training captions.

How it works

The node takes an IMAGE (from a Load Image node), converts it from a tensor into a plain PIL image, and injects it into the last user message of a chat. Then it calls the model's chat() with your sampling settings. Your half of the chat is:

• user_promptA - the instruction ("Describe this image in detail as a prompt for Flux")
• user_promptB - an optional assistant turn, i.e. an example answer that shapes the model's style. Fill both in and the node treats B as a prior response.

The image input is optional, by the way - with no image wired in, the node will happily answer a plain text question. Useful if you want a single VLM node for both captioning and quick prompts.

The two outputs and the keyword toggle

• responseText - the description. This is the one that goes into your text encoder.
• keywordsText - a comma-separated tag list, but only if you flip extract_keywords to true. Turning it on runs a second generation that asks the model to pull the key words out of its own answer. That's your tag output for SDXL or anime checkpoints that eat Danbooru-style tags.

The sampling knobs are the same quartet as the rest of the pack: max_new_tokens (default 300), temperature (0.5), top_p (0.8), top_k (50). A seed is here too, and unlike the text nodes this one actually applies it (when it's not zero) - set it if you want reproducible captions.

What it costs

An 8B VLM in fp16 is not small. The README's guidance is a "high-performance GPU," and in practice you want to keep your diffusion model in mind while it's resident - this node happily cohabits but it does want a real chunk of VRAM. It also calls .cuda() outright, so there's no CPU fallback: no CUDA GPU, no dice. First run downloads the full model into ComfyUI/models/MiniCPM/, so budget for a multi-GB pull on a stable connection.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/CY-CHENYUE/ComfyUI-MiniCPM-Plus

Or ComfyUI Manager → search "MiniCPM-Plus" → Install, then restart. First execution auto-installs the dependency list (transformers, accelerate, bitsandbytes, optimum, decord, …) and downloads the model - both are slow, and the pip install touches your shared environment, so restart before blaming other nodes.