Grok Image Understand

Grok Image Understand is the vision side of the pack's Grok trio: you hand it an image, it asks xAI's vision model what's going on, and hands the description back as a STRING - which you can then feed into a prompt, a caption field, or a classification branch in your workflow. It turns "what does this image contain" into a graph question instead of a manual look-and-type step.

The practical uses stack up fast. Auto-captioning a batch before you re-prompt or re-inpaint. Turning a reference image into a written description you can feed to a different generator. Or the meta workflow: generate an image, have Grok describe it, then use that description to refine the next generation. If you've ever written "describe this image" prompts by hand, this automates the loop. And because it's an API call, it doesn't care about your GPU - a 6GB card that can barely run a sampler can still caption 4K images all day.

How it works

It sends your image(s) to the xAI vision model (default grok-4) as part of a chat request built from system_prompt, prompt, and the chat_template. Up to five images go in: the main image input plus image1–image4, useful for comparing variations or asking "what's different between these two?" It supports history for follow-up questions ("and what color is the car in the first one?") and extra JSON for new API fields. Two details worth knowing: image_send_mode chooses between the standard OpenAI-style image_url object and xAI's base64 url field (switch it if the API rejects your images), and image_detail controls vision resolution - high for detailed scrutiny, low/auto for speed and fewer tokens.

The inputs that matter

• api_key - your xAI key; blank falls back to XAI_API_KEY/GROK_API_KEY env vars or the pack's api_key.json.
• prompt - the question. "Describe this image in detail" is the classic, but specific questions get better, more useful answers.
• system_prompt - role and style for the reply, e.g. "you are a terse booru-style tagger."
• image - the image to analyze (plus image1–image4 for multi-image queries).

Three outputs: text (the answer), history, and payload (raw response for debugging).

Installing it

It's part of ComfyUI-YogurtNodes. Install via ComfyUI Manager (search "ComfyUI-YogurtNodes") or:

cd ComfyUI/custom_nodes
git clone https://github.com/yogurt7771/ComfyUI-YogurtNodes.git
cd ComfyUI-YogurtNodes
pip install -r requirements.txt

Restart ComfyUI; it's under "Yogurt Nodes". You'll need an xAI API key (console.x.ai) - the vision models are paid.

Where people get burned

The two knobs that cause silent trouble are image_send_mode and image_detail. If images fail to send or the API complains about the image field, flip image_send_mode from "openai" to "base64" (and back if you switch providers). Keep image_detail at high for anything you actually need described precisely - the cheaper settings save tokens but miss details, and the whole point of this node is catching what you'd miss by eye. Also be honest about cost: vision calls bill per image per question, so a batch of 500 frames is a real line item. And if answers come back stale or templated, check the chat_template - same trap as the other Grok nodes.