Molmo Vision-Language Model

Molmo is Allen Institute for AI's open vision-language model family, and it's genuinely good at grounding - pointing at objects in an image and describing them precisely. MolmoNode brings the 1B and 7B variants into this pack as full transformers loads. The catch is written right into the node: these are big models, and the memory-mode tooltips don't mince words about it. This is a node you reach for when you want the model's quality, not when you're on a laptop.

What it's for

Grounded, specific visual understanding and Q&A. Molmo is strong at referring to parts of an image - "the red car behind the fence" - in a way that feels more deliberate than older captioners. If your workflow needs fine-grained spatial descriptions or you just want a different flavor of VLM than the Qwen/MiniCPM options, this is the alternate.

How it works

It downloads the model you pick from Hugging Face (allenai/MolmoE-1B-0924, Molmo-7B-D, or Molmo-7B-O) and runs it through Transformers with the model's custom processor (trust_remote_code). The memory_mode dropdown sets the strategy: full precision, 8-bit, 4-bit, or 4-bit with CPU offload. The 8/4-bit modes use bitsandbytes quantization; the CPU-offload mode pushes layers to system RAM, which is the only realistic path under ~16GB.

The inputs that matter

• model_name - the model card's own warning is worth repeating: start with MolmoE-1B (the efficient MoE). The 7B models are better, but they're the ones that want 25GB+.
• memory_mode - the tooltips are honest: Full Precision wants 45GB+, 8-bit wants 25GB+, 4-bit wants 15GB+, 4-bit + CPU offload wants 12GB+. Pick the most aggressive that runs on your machine.
• max_new_tokens - start at the 200 default; the tooltip's advice to start small is correct because every token costs VRAM.
• temperature / top_p / top_k - standard sampling; the defaults are sane.
• use_autocast - mixed precision, default ON. The tooltip recommends keeping it on, and that's right.

Output

A single STRING - the model's answer to your prompt.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

First run downloads several GB and may need a Hugging Face login if the checkpoint is gated. This node needs the bitsandbytes backend for the quantized modes.

The honest take

The memory requirements aren't fearmongering - a full-precision 7B VLM on top of whatever you're already running in ComfyUI is a real ask. If you have the VRAM, this is a great model to have in the pack. If you don't, the 4-bit + CPU offload mode at 12GB is the only path, and it'll be slow. On modest hardware, Qwen2-VL 2B or MiniCPM-V are the practical picks; Molmo is for when you specifically want Molmo.