Image to Text

Image to Text is the node that looks at a picture and tells you what it sees. Feed it any IMAGE and you get back a natural-language caption as a plain string. In a ComfyUI graph it's the workhorse for auto-captioning - generating alt text, labeling a folder of images, building caption files for a training run, or just seeding a prompt with a description of an existing image instead of typing one.

Mechanically it's the same story as the rest of ComfyUI-Transformers: a thin wrapper around a Hugging Face pipeline. The node takes the first frame of your image tensor, converts it to PIL, loads a model from the Hub (trust_remote_code=True), runs the image-to-text pipeline, and returns result[0]["generated_text"]. The default model is Salesforce/blip-image-captioning-base - the "base" BLIP, which is about 111M parameters. It's small, it runs fast even on modest hardware, and its captions are decent for a generic model.

The inputs that matter

Only three, and you'll mostly touch two:

• image - the IMAGE tensor to caption. As with every node in this pack, only frame zero is processed; batch mode won't caption each frame.
• max_new_tokens - how long the caption can get, default 50, range 1–512. Bump it if you're describing detailed scenes and the captions come back cut short.
• model_name - the Hugging Face model ID, typed as a string (no dropdown). Default Salesforce/blip-image-captioning-base.

The one output, generated_text, is a STRING - drop it into a text display node, append it to a file, or feed it into a prompt as a conditioning seed.

How to install it

Install the pack, not the node - they ship together. Either ComfyUI Manager (search "ComfyUI-Transformers") or:

cd ComfyUI/custom_nodes
git clone https://github.com/kadirnar/ComfyUI-Transformers

Restart ComfyUI afterwards. The pack's requirements.txt brings in transformers>=4.42, torch, Pillow, numpy and friends. The model itself is downloaded from the Hub on first run - BLIP-base is only a few hundred MB, so this is one of the lighter first loads in the pack.

Where people get burned

The honest take: don't train on raw BLIP captions. The training community has been saying for years that manual captions beat BLIP output for LoRA quality, and it's true - BLIP-base writes generic, pattern-y captions ("a person standing in front of a building") that leave out exactly the details a good tagger would keep. Use this node for alt text, quick labeling, or pipeline prototyping, and treat its output as a starting draft if it's heading into training data.

One more quirk: because model_name is a free string, a typo means the model fails to load. Copy IDs carefully from Hugging Face. And if a caption looks weird, remember the default token cap is 50 - short answers are by design, not a bug.