Replicate cuuupid/glm-4v-9b

GLM-4V-9B is a vision-language model from Zhipu AI - you give it an image and a prompt, and it reads the image and responds in text. Think detailed captioning, visual question answering, reading text out of a picture, describing a scene for a downstream prompt. It's a newer, more capable generation than the old BLIP-2 captioner, so its answers are longer, more reasoned, and better at nuance. This node runs it from ComfyUI via ComfyUI-Replicate.

It runs on Replicate's cloud. A 9B multimodal model can technically run locally, but this node deliberately doesn't - it offloads to Replicate so you don't manage the weights or the VRAM.

How it works

GLM-4V pairs a vision encoder with the GLM-4 language model, so it can attend to what's in the image while it writes. You give it an instruction ("Describe this image in detail" or "What's written on the sign?") plus the image, and it answers. The node sends both to Replicate and returns the reply as a STRING. No local compute, pay per run.

Inputs and outputs that matter

Two required inputs - an IMAGE and a prompt (your instruction/question). Output is a single STRING.

• prompt - required, and it's the steering wheel. Be specific about what you want: a caption, an answer, a structured list, a prompt rewrite based on the image.
• max_length (512) - caps the response length, up to 8192. Raise it if you want a long, detailed description.
• top_k (1) - sampling. At 1 it's essentially greedy and deterministic; raise it for more varied phrasing.

force_rerun forces a fresh response even when the image and prompt are unchanged.

How to install it

ComfyUI Manager: search ComfyUI-Replicate, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart, and set your Replicate token before launch:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Token: replicate.com/account/api-tokens.

Common issues

This node returns plain text - the "simple output" case the pack's README supports cleanly, so no audio/multi-output caveats apply.

The most common non-obvious issue is length: if the description gets cut off mid-sentence, raise max_length. If it's too rambly or too repetitive, drop it. Because prompt is required, an empty prompt won't get you a useful caption - actually tell it what to do ("Describe this image for a Stable Diffusion prompt").

Otherwise the usual: a missing REPLICATE_API_TOKEN is the first-run failure, and every call costs money on Replicate. As a captioner it's stronger than BLIP-2 but also pricier and slower per call - if you just need a five-word "what is this," BLIP-2 is the cheaper tool; reach for GLM-4V when you actually need the model to reason about the image.