Molmo 7B D bnb 4bit

Molmo 7B D bnb 4bit is a "one node does a whole job" deal: drop an image in, get a text caption out, and - the part the author actually built it for - feed that text straight back in as a prompt to generate a new image. No API, no key, no second app. It runs a real vision-language model locally, Allen Institute for AI's Molmo 7B-D, pre-quantized to 4-bit so it fits on a normal card. The whole image-to-text-to-image loop lives inside your ComfyUI graph.

What it is

Under the hood it's Molmo 7B-D (a Qwen2-7B language core with an OpenAI CLIP visual backbone, per the author) quantized with bitsandbytes by cyan2k. The node loads that model, describes whatever image you hand it, and returns a plain STRING. You'll find it under the Molmo category in the node menu.

The pack author framed it as the painless version of this job. His launch post was basically "people keep breaking their installs fighting JoyCaption's dependencies, so I built a node where all of it happens automatically." That's the pitch in one line: one node, auto-installed dependencies, auto-downloaded model, restart once, done.

Honest caveat before you get excited: it's not the best captioner on the shelf. The same thread has a tester saying it trailed some Qwen2-VL finetunes on quality. For casual captioning and prompt-building it's perfectly usable - just know it's a convenience pick, not an SOTA one.

How it works

Everything is lazy-loaded. On first run the node pip-installs its requirements (transformers, accelerate, bitsandbytes, torchvision, huggingface_hub, numpy - and, oddly, tensorflow, which nothing here uses), writes a status file, and then deliberately raises an error telling you to restart ComfyUI. After the restart it downloads the model via snapshot_download into ComfyUI/models/Molmo/molmo-7B-D-bnb-4bit. A 4-bit 7B lands around 5 GB, so the first run is a download-and-settle wait no matter how fast your connection is.

Inference is straightforward: it converts your image to PIL, pads it onto a black or white backdrop chosen from the image's average brightness (an automatic way to dodge alpha-channel weirdness), feeds it to the processor with one of two canned prompts, and samples a caption. prompt_type picks the canned prompt; a non-empty custom_prompt overrides it entirely.

The inputs that matter

• image - any image tensor in your graph.
• prompt_type - Describe for a general caption, Detailed Analysis for style/theme/scene/composition/lighting breakdowns.
• custom_prompt - leave blank to use prompt_type; fill it to ask your own question ("what color is the subject's hair?").
• seed - set 0 and every run gives you a fresh random caption; set a value to make a run reproducible.
• max_new_tokens - default 350, plenty for a description.
• temperature / top_k / top_p - sampling knobs. The defaults (0.6 / 40 / 0.9) are fine; don't touch them until you have a reason.
• unload_model_after_generation - default on, and it matters.

The output

One STRING, the generated caption. Wire it into a CLIP Text Encode to build a prompt from a reference image, into a Save Text node to log captions, or into an auto-captioning workflow. The model stays in memory only as long as the node runs.

Installing it

The easy way: ComfyUI Manager → search ComfyUI-Molmo → install → restart. The manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/CY-CHENYUE/ComfyUI-Molmo

Then restart ComfyUI. First run installs dependencies and pulls the ~5 GB model; the README notes you may need to restart again after the dependency install, and that Chinese users can grab the model from a quark netdisk mirror if HuggingFace is unreachable. CUDA is the intended path - the code will fall back to CPU, but you don't want that.

Where people get burned

The auto-installer is blunt. It pip-installs everything it lists - including that needless tensorflow and a very new torchvision - into your shared ComfyUI Python environment, which is exactly the setup that causes cross-node dependency fights in this ecosystem. If other nodes stop loading after the install, restart first; that's the documented step and it fixes most of it.

The other trap is unload_model_after_generation. It's on by default, so every single run ends by wiping the model and clearing CUDA cache. Great if you're about to hit a big image-gen step and need the VRAM. But if you're captioning several images in a row, you're re-loading a 5 GB model each time. Flip it off for batch work; you'll get the VRAM back when the node finishes anyway. And remember: it returns prose captions, not danbooru tags - if you wanted WD14-style tag lists, this isn't that tool.