MiniCPM 🤖👁️(WildPromptor)

This is the odd node out in the pack - a full vision-language model bolted onto a prompt-wildcard toolkit. Feed MiniCPM an image and a question ("describe this," "what's the lighting?") and it writes back a description you can use as a prompt. Or run it text-only to expand a rough idea. It's local: the model runs on your machine, no API, no key.

It's part of WildPromptor by 1038lab - and if a prompt pack shipping a bundled VLM seems ambitious, note that the same author's ComfyUI-RMBG pack bundles a dozen segmentation and matting models behind one interface. Fat, batteries-included packs are their style.

Why it's here

Image-to-prompt captioning is the backbone of a lot of modern workflows - img2img, style transfer, building a text prompt from a reference photo. And LLM-assisted prompting generally has gone from novelty to standard practice, roughly a twentyfold rise in community usage over three years. A caption model that lives in the graph, right next to the nodes that assemble prompts, closes the loop: reference image in, structured prompt out, straight into your list/concat chain.

How it works and the inputs that matter

MiniCPM-V is a compact open vision-language model; this node runs the int4-quantized builds of it locally. The inputs:

• text (required, multiline) - your instruction or question; with an image attached it's "how should I describe this," text-only it's a prompt to expand
• model - which build to load: MiniCPM-V-2_6-int4 (default), MiniCPM-Llama3-V-2_5-int4, MiniCPM-V-4-int4, or MiniCPM-V-4_5-int4
• language - Auto, English, Chinese, Japanese, or Korean output
• temperature - 0 to 1; lower is literal and repeatable, higher is looser and more creative
• seed - for reproducibility
• image (optional, IMAGE) - attach a reference to caption it; leave it off for pure text work

The output is a single STRING - the model's text. Wire it into a CLIP Text Encode, or into Prompt Concat / Prompt Builder to frame it with your own prefix and suffix.

The catch: this one has real weight

Every other node in this pack is a lightweight text shuffler. This is not. The first time you pick a model, it downloads the weights from HuggingFace - a multi-gigabyte pull per variant - and it needs a GPU to run at any speed. The int4 quantization also drags in a quant/vision dependency stack (the usual bitsandbytes / accelerate territory), which is a known source of install pain, especially on Windows. If you only came for the wildcard dropdowns, you never need to touch this node, and skipping it keeps the pack featherweight.

Practical note on output length: caption models love to write paragraphs, and long prompts hurt you on modern encoders - past ~75–100 effective tokens they start to drift. Ask MiniCPM for a concise description, or trim its output before you generate.

Installing it

ComfyUI Manager: search ComfyUI-WildPromptor, install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/1038lab/ComfyUI-WildPromptor

then restart. The node is under 🧪AILab/🤖AI. The MiniCPM weights download on first use, separately from the node install.

Common issues

• First run hangs or takes forever - it's downloading multi-GB weights. Watch the console; it's a one-time pull per model.
• Errors about quantization / bitsandbytes / CUDA - int4 loading needs a compatible GPU and the right quant dependencies. This is the usual VLM install friction and hits Windows hardest.
• Out-of-memory - int4 keeps it modest but it's still a real model; close other GPU workloads, and prefer a smaller variant if a larger one won't fit.
• Caption is too long and dilutes results - ask for something concise or edit it down before feeding an encoder; verbose captions overrun the attention window on current models.