ComfyUI_ImageToText

The name is the whole pitch: drop an image in, get a natural-language description out. No API, no key, no cloud upload - ComfyUI_ImageToText runs a small vision-language model on your own machine and hands you a sentence you can paste straight into a prompt box. If you've ever stared at a reference image and thought "I have no idea what to type," this is the node that types it for you.

It's built on vikhyatk/moondream2, the same lightweight ~1.6B vision-language model family that SUPIR uses to caption images before restoration. Being small is the point: moondream was designed to run on hardware a real LLM would laugh at. The author, SoftMeng, is a Chinese developer with a whole shelf of ComfyUI utilities (a translator, a prompt randomizer, the Mexx poster nodes), and this one is his simplest - a ~40-line wrapper around the model's built-in Describe this image. question.

Why you'd reach for it

The obvious workflow is reverse-engineering a prompt from an image you like. Feed the description into a CLIPTextEncode node as the positive prompt and you get a decent starting point for a new generation - or flip it into the negative prompt to steer away from what the image shows, a trick that's been popular since the img2txt days. It's also handy for captioning datasets: the repo ships BatchImageToText.py, a standalone script that walks a folder and writes a matching .txt next to every image.

One honest caveat: this gives you prose, not danbooru tags. Moondream writes "a ginger cat with white paws sitting on a snowy field," it doesn't emit 1girl, animal ears, snow. For models that were trained on tag vocabularies (anything danbooru-flavored), a WD14-style tagger will serve you better; for the newer LLM-encoded models that were trained on real captions, a natural-language description is exactly the right shape. Pick the tool to match the model.

How it works

Mechanically it's unglamorous in the best way. Your IMAGE tensor gets converted to a PIL image, and the node calls moondream's encode_image then answer_question with a hardcoded "Describe this image." The model and tokenizer are pulled from HuggingFace with trust_remote_code=True on a pinned revision (2024-04-02), so you always get that old snapshot, never a surprise update.

The inputs are exactly two:

• images (IMAGE) - wire up any image source. One catch: it only actually describes the first image in the batch, even though it accepts a batch.
• log_prompt (Yes/No, default Yes) - prints the description to the ComfyUI console.

The single output, text_positive (STRING), goes into CLIPTextEncode's text input, or into a ShowText node if you just want to read it. The bundled example workflow is LoadImage → this node → ShowText.

Installing it

ComfyUI Manager can find it by searching ComfyUI_ImageToText. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/SoftMeng/ComfyUI_ImageToText
# then restart ComfyUI (Manager handles deps; manual installs may need:)
cd ComfyUI_ImageToText && pip install -r requirements.txt

Its requirements.txt is just transformers, timm, and einops. There's no model file to download by hand - it auto-downloads from HuggingFace on first run, a couple of gigabytes you'll want internet for.

Where people get burned

The model is loaded every time the node executes - there's no caching inside the node, so each run re-reads it from disk. Fine for one-off reverse-prompting, painful if you're looping it over a folder in a workflow. That's what the batch script is for.

Other real traps:

• First-run download failure. If you're somewhere HuggingFace is blocked (common for the author's own Chinese audience), the fetch just dies. Set HF_ENDPOINT=https://hf-mirror.com and retry.
• ModuleNotFoundError: No module named 'timm' or transformers version conflicts. transformers is a heavy dependency shared by half the custom-node ecosystem, so pin-battles are the norm - the classic ComfyUI dependency hell.
• trust_remote_code=True is worth a raised eyebrow in a post-LLMVISION world. It's the standard way to run a model with custom architecture code, and moondream is legit, but know that remote code is executing.

Is it the fanciest tool in the drawer? No. But for a one-way image-to-prompt trip on modest hardware, it's honest, dependency-light, and done. The name isn't a lie.