Loader Image to Text Model 🐼

What it is

The loader that everything else in this pack's captioning half depends on. Image2Text and Image2TextWithTags both need an IMAGE2TEXT_MODEL wired in, and this is the only node that produces one. Pick your vision-language backbone here; everything downstream just uses whatever you loaded.

How it works

It's a straight loader: pick a model name from the dropdown, pick a device, decide whether to trade speed for memory, and it hands back a loaded model object. The interesting part is the menu itself - eight quite different models, spanning small dedicated captioners up to full 7-8B chat-style vision-language models, plus one that isn't a captioner at all.

Inputs and outputs

• model - eight choices, default moondream2:
  • moondream1 / moondream2 - small, fast VLMs purpose-built for captioning. The README's own advice: moondream1 gives richer scene detail but can ramble; moondream2 is tighter and more accurate, especially for people. moondream2 as the default is the reasonable starting point for most users.
  • deepseek-vl-1.3b-chat / deepseek-vl-7b-chat - DeepSeek's vision-language chat models, in two sizes. The 1.3B is the practical choice on a consumer card; the 7B needs real VRAM.
  • bunny-llama3-8b-v - BAAI's Bunny multimodal model built on Llama-3-8B. The heaviest option on this list.
  • internlm-xcomposer2-vl-7b - a larger 7B vision-language model, also on the heavy end.
  • uform-qwen - unum-cloud's compact VLM (uform-gen2-qwen-500m), small and fast.
  • wd-swinv2-tagger-v3 - the odd one out. Not a chat-style captioner at all; it's a Danbooru-style tagger (SwinV2 backbone, same lineage as the well-known WD14 tagger). Load this one if you want tag output rather than a sentence, or if you're feeding Image2TextWithTags's tagging half.
• device - cuda by default, cpu available. Bigger models on CPU will work but will be slow; the pack's own text-generation loader (Loader Text to Prompt Model) defaults to CPU precisely because its model is small enough to tolerate it - the 7-8B vision models here are not in that category.
• low_memory - off by default. Turn it on if you're loading one of the larger models and running short on VRAM; expect a speed tradeoff in exchange.

Output: IMAGE2TEXT_MODEL, wired into Image2Text or Image2TextWithTags.

Installing it

Same pack as every node here. ComfyUI Manager: search "Comfyui_image2prompt." Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/zhongpei/Comfyui_image2prompt

The README's own clone command has a typo (hyphenated URL, doesn't resolve) - use the underscored one above, then restart ComfyUI.

Models download automatically from Hugging Face into ComfyUI/models/image2text/ the first time you select and run them. If the auto-download doesn't kick in, the README documents pulling them manually with huggingface-cli download, and sets HF_ENDPOINT=https://hf-mirror.com for anyone downloading from behind the Great Firewall.

Common issues

Picking a model too big for your card. bunny-llama3-8b-v, internlm-xcomposer2-vl-7b, and deepseek-vl-7b-chat are all genuinely large - start with moondream2 or deepseek-vl-1.3b-chat and only move up if you have the VRAM to spare and actually need the extra quality.

First selection is slow. Switching the dropdown to a model you haven't loaded before triggers a multi-gigabyte download before anything runs - this isn't a hang, check your console.

Real-world reports of workflow files not loading cleanly for this pack have shown up in community threads; if a shared workflow using this node won't open, check that your installed node version's class names actually match what the JSON expects before assuming your setup is broken.