MiniCPM-V (Advanced)

If you've ever wanted your workflow to look at an image and tell you what's actually in it - not tags, sentences - this is the node. MiniCPM-V (Advanced) runs OpenBMB's MiniCPM-V-4.5 vision-language model fully locally, straight through HuggingFace Transformers. The name is a lie only in the sense that "advanced" oversells the difficulty: it's the full-parameter version of the same pack's plain MiniCPM-V node, and it's the one I'd actually reach for when caption quality matters.

The name is also not an API call. Nothing here phones home for inference - the model downloads from HuggingFace on first run and then lives in ComfyUI/models/LLM. Given the ComfyUI ecosystem's history with vision-LLM nodes that turned out to be malware, "downloads a HuggingFace checkpoint, runs locally" is exactly the behavior you want.

What it's good for

Natural-language captions for training data, mostly. WD14 gives you tags; MiniCPM-V gives you descriptions - "a woman in a yellow raincoat walks a corgi across a wet street at dusk" - which is what Flux-class and other LLM-encoded models actually want for LoRA training. BLIP is widely considered unusable for that job; MiniCPM-V-4.5 is a genuinely better class of model, and it runs on the GPU you already have. It's also fine for quick visual analysis: describe, analyze, explain, and more, all behind a single enum.

How it works

The node loads openbmb/MiniCPM-V-4_5 (or the int4 variant) with AutoModel.from_pretrained, trust_remote_code=True, SDPA attention, and bf16 if your GPU is Ampere-or-newer (fp16 otherwise). First run downloads a few GB and loads the model into VRAM - that's the slow part, not the captioning. Then it runs the model's chat() method with your system prompt and image, sampling with your chosen parameters. Feed it a video input and it uniformly samples up to video_max_num_frames frames and treats them as a multi-frame question.

The inputs that matter

Everything is optional, which is the trap: defaults are sane but worth knowing.

• model - four choices: MiniCPM-V-4.5-int4 (default), MiniCPM-V-4-int4, full MiniCPM-V-4.5, full MiniCPM-V-4. Start with the default; it's the VRAM/quality sweet spot. 4.5 full needs ~12GB+, int4 runs in 8.
• preset_prompt - 21 built-in prompts, from Describe/Caption to Style/Location, plus Video variants. custom_prompt overrides it whenever it's non-empty, so you can leave the preset as a fallback.
• system_prompt - the Advanced node's superpower; the plain version hardcodes this. Tune it to force terse captions or a specific voice.
• max_new_tokens (1024), temperature (0.7), top_p (0.9), top_k (100), repetition_penalty (1.1) - the usual sampling knobs. For captioning, drop temperature toward 0.6 and raise repetition penalty to 1.2 if you're seeing loops.
• device (Auto/GPU/CPU) and memory_management (Keep in Memory / Clear After Run / Global Cache) - set Keep in Memory if you're captioning a batch; the reload penalty on every run is real.
• video_max_num_frames (64) and video_max_slice_nums (2) only matter for video input. Slice numbers control MiniCPM's high-res image slicing.

Outputs

Two strings, and it's easy to grab the wrong one. PROMPT is the prompt that was actually sent (preset or custom). STRING is the model's answer - the caption you want. Wire STRING into a text display or a caption-saving node; grab PROMPT if you want to see or reuse what got asked.

Installing

ComfyUI Manager (search "MiniCPM") or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/1038lab/ComfyUI-MiniCPM
cd ComfyUI-MiniCPM
pip install -r requirements.txt

Restart, then drag it in from the 🧪AILab/📝MiniCPM category. The model auto-downloads on first use - grab a coffee. If you hit CUDA out-of-memory, switch to MiniCPM-V-4-int4 or drop to the GGUF variant of this pack.