Load MiniCPM-o Model

This is the node that makes the whole ComfyUI-MiniCPM-o pack exist. Nothing else in the pack works until you've got a loaded MiniCPM-o model and its tokenizer wired out of this thing - it's the Load Checkpoint of the MiniCPM world, except the "checkpoint" is an 8B-parameter omni-model from OpenBMB that can see.

If you've been feeding screenshots to a website LLM to reverse a prompt, you already understand why this exists. MiniCPM-o 2.6 reads an image and describes it back to you as text, which means it slots into that whole "have an LLM describe the input image so the video/diffusion model understands it" workflow people keep building around LTX and the LLM-encoded image models. The catch is the model itself has to live on your machine, and this node is how ComfyUI gets it there.

How it works

The node does four things in order. First, it looks for a local folder at ComfyUI/models/MiniCPM/MiniCPM-o-2_6/ and errors out if it's not there - it does not auto-download from Hugging Face at runtime. Second, it does a sneaky bit of housekeeping: it copies image_processing_minicpmv.py from that folder into your Hugging Face transformers modules cache, because the model's remote code expects that file in a specific cache location. Third, it loads the model with trust_remote_code=True - the model folder is Python, not just weights. Finally, it returns the model and tokenizer as a MODEL/TOKENIZER pair for the other two nodes in this pack.

What actually loads is a bf16 model on cuda (or fp32 on cpu, which is slow enough to feel like a mistake). Attention uses sdpa, and device_map is a single device - no clever offload split, so the whole thing needs to fit in one card's VRAM. Plan for a 24 GB card to be comfortable. The author's been upfront that the stock model "requires a larger amount of VRAM" and there's no built-in quantization in this pack.

Inputs that matter

The dropdown model_name has exactly one choice, MiniCPM-o-2_6 - that's hardcoded, so don't go hunting for a list of supported models. The two toggles you actually touch:

• init_vision (default on) - needed for every image task in this pack. Leave it on.
• init_audio / init_tts (both off) - these load MiniCPM-o's audio and speech-synthesis components, which cost extra VRAM. Since the pack only ships image nodes so far, leave them off unless you're planning something exotic.

device is just cuda or cpu. Pick cuda.

Installing

Easiest path is ComfyUI Manager - search "MiniCPM-o" and hit install. Manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/CY-CHENYUE/ComfyUI-MiniCPM-o.git
# then install deps with ComfyUI's own python:
python_embeded\python.exe -m pip install -r requirements.txt   # Windows
/path/to/ComfyUI/python/bin/python -m pip install -r requirements.txt  # Linux/mac

The requirements.txt pulls librosa, accelerate, vector-quantize-pytorch, and vocos - mostly needed for the audio side of the model, but install them anyway.

Then the real work: download the whole openbmb/MiniCPM-o-2_6 repo from Hugging Face and drop it in as models/MiniCPM/MiniCPM-o-2_6/ - config files, modeling scripts, and safetensors together, filenames untouched.

Where people get burned

The "no model found" error at models/MiniCPM/MiniCPM-o-2_6 is the most common complaint - you must get the full folder, not just the safetensors. If you see an import error about image_processing_minicpmv, that's the cache-copy step failing; the node tries a fallback cache path, but on odd HF setups you may need to clear ~/.cache/huggingface/modules and retry.

And one honest word of caution: trust_remote_code=True means whatever Python is in that model folder executes on your machine. Download it from the official openbmb repo, not a random mirror. The ComfyUI ecosystem has a documented history of malicious custom nodes, so treat remote-code model folders the same way you'd treat a sketchy download.