MS kosmos-2 Interrogator

Most image-to-text nodes just hand you a caption and stop there. This one is built on Microsoft's kosmos-2, a "grounding" multimodal model - meaning it doesn't just describe an image, it can tie the things it's talking about back to actual regions in the image. Feed it a photo and it'll write you a description, pull out keyword phrases, and hand back a mask of the elements it recognized. That grounding is the whole reason to pick this over a plain tagger: it's genuinely good at reading text baked into an image and recognizing specific, named things (people, landmarks, objects) rather than just generic visual attributes.

What it's doing under the hood

kosmos-2 is a text-and-image-to-text transformer, and the way you steer it is with a lead-in prompt rather than a question. The author's own example: prompt it with "An image of" and you get a plain description; prompt it with "Very detailed, an image of" and it writes noticeably more. It's not a Q&A model - it's completing a sentence you start, using the image as context. That's also why the model recognizes specific people and on-image text unusually well for its size: it was trained to ground its output in actual image regions, not just guess at a generic caption.

The inputs and outputs that matter

• image - what you want described.
• prompt (default "An image of") - the lead-in text that steers the output. Change the wording, change the level of detail and framing you get back.
• model - currently just microsoft/kosmos-2-patch14-224, the one model this node supports.
• device - cpu (default) or gpu. Slower on CPU, but GPU mode eats real VRAM (see below), so this is a genuine trade-off, not just a speed knob.
• strip_prompt (default true) - since the model literally continues your prompt text, the raw output would start with whatever you typed ("An image of a dog running..."). This strips that lead-in back out so you get a clean description rather than your own prompt echoed at the front of it.

Three outputs: description (the full caption, a plain STRING), keywords (extracted key phrases, also STRING), and mask - a MASK highlighting the regions the model actually recognized. That mask is the grounding feature made concrete: it's not a generic saliency map, it's tied to the specific elements the model named in its description. Wire description into a CLIP Text Encode for prompt-from-image workflows, or use mask to feed a targeted inpaint on just the thing the model identified.

Installing it

Search ComfyUI-Hangover-Nodes in ComfyUI Manager, or clone it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/Hangover3832/ComfyUI-Hangover-Nodes

Restart ComfyUI. On first use, the model downloads automatically from HuggingFace - about 6GB, so budget time and bandwidth for it. If you'd rather grab it yourself (or your network doesn't play nice with the automatic download), manually download the microsoft/kosmos-2-patch14-224 repo and place it at ComfyUI/models/kosmos2/kosmos-2-patch14-224/, containing added_tokens.json, config.json, generation_config.json, model.safetensors, preprocessor_config.json, sentencepiece.bpe.model, special_tokens_map.json, tokenizer.json, and tokenizer_config.json. That base folder is also configurable via extra_model_paths.yaml if you keep models on another drive.

Is it actually good, and what's it for

The real-world use case people report is dataset captioning for LoRA/DreamBooth training - kosmos-2 shows up in community comparisons against WD14 tagging and manual captions as one legitimate option among several, particularly valued for reading text and recognizing specific subjects rather than just tagging generic attributes. It's a solid, honest tool for that. Just don't expect it to be the newest thing on the block: this pack is marked by its author as no longer maintained, and by 2026 captioning has moved on to newer vision-language models. Fine for a grounded caption already sitting in your node tree; worth comparing against something current if you're building a serious captioning pipeline from scratch.

Common issues

GPU mode plus a diffusion model already loaded is the classic OOM trap. The README is upfront about it: kosmos-2 eats roughly 6GB of VRAM on top of whatever else is resident. If you're running it in the same graph as an SDXL or Flux checkpoint on a card that's already tight, either switch device to cpu or make sure kosmos-2 runs (and unloads) before your main model loads. CPU mode is slower but sidesteps the VRAM fight entirely, and for a one-off caption per image it's usually fine.

The other likely snag is the first-run download itself - a 6GB pull from HuggingFace can stall or fail on a flaky connection. If it hangs, don't just retry blindly; grab the files manually from the model page and drop them in models/kosmos2/kosmos-2-patch14-224/ as described above, which sidesteps the in-node downloader entirely.