AI4ArtsEd Image Analysis

AI4ArtsEd Image Analysis sends whatever image you give it to a multimodal LLM and returns the answer as text. It's the pack's eye: feed it any IMAGE, ask a question in the prompt field, and get back a description, a judgment, or an extracted fact. The same "image-to-text-to-image" loop that turned into a whole genre of ComfyUI workflows - caption the image, feed the caption back in as a prompt - starts with exactly this kind of node.

The mechanism is simple and solid. The node takes the first frame of your IMAGE tensor, converts it to a JPEG, base64-encodes it, and posts it to the model as a multimodal request. Locally it hits Ollama (llava:7b, 13b, or 34b); via OpenRouter it can reach the same llava variants hosted. The model dropdown is hardcoded to those six entries, which keeps things predictable but also means you're limited to llava flavors - no GPT-4V here despite what the README hints at.

Outputs are where it gets clever-odd:

• output_str - the model's raw text answer. This is the one you'll actually use.
• output_float, output_int, output_binary - the same text run through regex parsers. If you prompt the model to answer "with a number between 0 and 1," the float output is genuinely useful. If you don't, they're all zero/false.

That pattern - one text output plus three convenience-parsed casts - is worth knowing because it lets a single vision call drive conditional logic downstream. Ask "does this image show a cat?" and output_binary becomes your branch condition.

The inputs that matter

• image - any IMAGE. First frame only.
• prompt - your question or instruction to the model.
• model - local/llava:* or openrouter/llava:*.
• system_prompt (optional) - steer the model's role.
• api_key (optional) - OpenRouter key; falls back to openrouter.key.
• debug - prints the request/response to the console.
• unload_model - evicts the local model from memory after the call (useful if you're swapping between a 34B vision model and your generation model on limited VRAM).

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/joeriben/ai4artsed_comfyui_nodes
# restart ComfyUI

Or search "AI4ArtsEd" in ComfyUI Manager. For local inference, install Ollama and pull a vision model:

ollama pull llava:13b

For hosted, you need an OpenRouter key.

Common issues

Local calls fail. The node talks to Ollama at localhost:11434 via plain HTTP. If Ollama isn't running or the model isn't pulled, you get an error string that starts with [Error Ollama]. Verify with ollama list.

Output is cached. No seed, no IS_CHANGED here - change an input (or the image) to force a re-run. Same as ai4artsed_prompt_interception.

output_binary gives surprising results. It's true if the text contains "true", "yes", or a non-zero number. If your model answers "No, it's a dog," the word "yes" isn't there but a stray number could flip the flag. Read the logic before you wire it into a switch.

VRAM pressure. llava:34b is a heavy local model. unload_model to "yes" helps when you're juggling it with a diffusion model on one GPU.

It's a genuinely handy node - vision analysis inside the graph is a capability that unlocks a lot of conditional workflows. Just remember you're on llava, so prompt it like a reasonable assistant and don't expect frontier-model reasoning.