Griptape Run: Image Description

This is the flagship of the pack, and the workflow in the repo's README shows exactly why. Griptape Run: Image Description takes an image from your graph and asks a vision-capable LLM about it - and the answer comes back as text you can feed straight into another prompt. The README example runs three of these on different images, merges the descriptions, and uses them as the prompt for a new image generation. Description → generation, all inside one graph.

It's the natural place for ComfyUI to meet LLMs: ComfyUI is great at producing images but knows nothing about them. This node is how you get an image's content into text form so an agent (or another image model) can reason over it.

How it works

The node converts the incoming IMAGE tensor to base64, loads it as an image artifact, and runs a PromptTask with your STRING as the question. The model doing the answering is whatever your agent's prompt driver is - so if that driver isn't vision-capable, you'll get nonsense. Connect an agent whose config uses a multimodal model (OpenAI, Anthropic, Gemini, local vision models via Ollama, etc.).

A couple of quirks baked in by the author: for Bedrock and Anthropic drivers, an empty prompt gets defaulted to "Describe this image" because those APIs reject empty prompts, and for Groq the node temporarily strips rulesets (Groq doesn't support them). For more than two images it builds a small workflow that concatenates per-image answers.

The inputs

• STRING - your question. "Describe this image in detail" is the boring default that works.
• image - the IMAGE from anywhere in your graph.
• agent - optional; the agent whose model does the seeing. Leave empty and you get the default config.
• input_string / key_value_replacement - prompt appendix and template fill.

Outputs: OUTPUT (the description) and AGENT.

Installing

ComfyUI Manager → search "Griptape" → Install, or:

cd ComfyUI/custom_nodes
git clone https://github.com/griptape-ai/ComfyUI-Griptape

Restart ComfyUI. Heavy deps (griptape[all], python-dotenv), and you need a vision-capable model configured.

Common gotchas

The pack-wide environmental issues apply: torch version conflicts on Nvidia (reinstall with the cu121 index) and stale-griptape ImportErrors fixed with python -m pip install griptape -U.

The image-specific trap is using a text-only model - you'll get an error or gibberish rather than a useful description. And cost: each description is a paid API call on the hosted path, so the "run three of these per workflow" pattern from the README adds up fast. Local vision models via Ollama are the way to iterate cheaply.