GLM识图生成提示词

The image-to-prompt node that runs on nothing

GLM_Vision_ImageToPrompt is the vision member of the ComfyUI-GLM4 pack. You feed it an image, it sends it to Zhipu AI's GLM-4V model, and it returns a detailed English description prompt you can wire straight into a CLIP Text Encode for img2img, or into an image-to-video pipeline. And the default model, glm-4v-flash, sits on Zhipu's free Flash tier - so this is a captioner that costs you nothing and demands no VRAM at all. If you've got a machine too weak to run JoyCaption or Florence-2 locally, that's a genuinely useful door.

How it works

The important 2025 update in the README: the node now converts a ComfyUI IMAGE tensor to a base64 PNG inside itself, so you don't need a separate conversion node in the graph anymore. It then sends the image plus a description instruction to the GLM-4V API and returns the model's text.

You've got three ways to hand it a picture, and the tooltip is blunt about it - choose one of three:

• image_input - the normal path. A ComfyUI IMAGE object straight from your graph (a Load Image, an img2img result, a video frame).
• image_base64 - a Base64-encoded image string, if your data comes in that shape.
• image_url - a remote URL to an image.

If more than one is connected, priority runs IMAGE → Base64 → URL. Give it none and you get an error string back: it needs at least one.

The other input that matters is image_prompt_preset, with three built-in instructions loaded from a text file: 通用高质量英文描述 (generic high-quality English description), Flux描述 (Flux description), and 视频描述 (video description - structured for image-to-video motion prompts, subject + scene + motion + camera language). The generic preset already appends quality tags like best quality, high resolution, 4k to whatever it writes. And prompt_override, if you fill it in, beats the preset - same escape hatch as the pack's chat node.

The output is GETPrompt, a plain STRING, which is your ready-made prompt. Feed it to CLIP Text Encode for FLUX (whose LLM encoder loves natural-language descriptions - that's exactly the preset format), or into an image-to-video workflow.

The honest tradeoff vs. local captioners

The KB's VLM-captioning section steers you to JoyCaption and Florence-2 for good reasons: they're local, offline, and uncensored, which matters when you're captioning a LoRA training set. This node is the API door - free tier, zero VRAM, works on anything - but your image leaves the machine and goes to Zhipu's servers, and their content filter applies. Don't send reference images you'd rather a cloud vendor didn't see. Also bear in mind the weakness every VLM shares: multi-subject attribution. Two people in frame and it'll likely mix up who's wearing what. Treat its output as a strong first draft and audit it for anything that matters.

Installing it

Same pack, one install, all three nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/msola-ht/ComfyUI-GLM4.git
cd ComfyUI-GLM4
pip install -r requirements.txt

Restart ComfyUI. Dependencies are zhipuai, Pillow, and numpy - nothing heavy. Grab a Zhipu API key from bigmodel.cn and put it in the node's api_key field, the ZHIPUAI_API_KEY env var, or the pack's config.json.

Gotchas

• No image connected, no output. The error string "必须提供图片URL、Base64数据或IMAGE对象" comes back as text, not a crash. Read your node's output.
• Stay on the free vision model. glm-4v-flash is the default and the free one. If someone has typed a paid model name in there, the free-tier story ends.
• The seed input is cosmetic again - same tooltip as the rest of the pack: it only affects node-internal randomness, not what GLM-4V generates. The API doesn't take a seed.
• Thin maintenance. One-person Chinese-language project; the changelog warns that Zhipu's library updates outran the plugin. It works today, but don't build your whole pipeline on it without a fallback.

For a zero-cost, zero-VRAM image-to-prompt step - especially aimed at FLUX and video workflows - this is a solid little node. Just remember the image you feed it is the price you're paying.