🦙 Ollama Image Describer 🦙

This is the node people mean when they say "hook ComfyUI up to a local LLM to describe an image." It sends whatever image you feed it to a vision model running on your own machine via Ollama - LLaVA, LLaVA-Llama3, MiniCPM-V, Moondream, whatever you've pulled - and gets back a text description. No OpenAI key, no cloud call, nothing leaves your box. That's the whole pitch, and it's a good one if you already have Ollama running for something else.

Where it earns a spot in a real workflow: feeding a description into a prompt for img2img, auto-tagging a folder of renders, or just asking "what's actually in this image" when you're debugging a batch that came out weird. It's a general-purpose describer, not a purpose-built captioner for LoRA training - if that's your goal, the community's actual go-to for that job is JoyCaption or Florence 2 (see the training-captions notes below), which are tuned specifically for terse, trigger-word-friendly dataset captions rather than the fuller prose a chat-style vision model like LLaVA tends to produce.

How it works

The node talks to Ollama's local HTTP API (default http://localhost:11434), not to Ollama's Python package baked into ComfyUI - meaning Ollama has to be installed and running separately, with the model already pulled, before this node will do anything. You pick a model from the dropdown or type a custom_model name if it's not in the list; the node then ships your images and your prompt to that endpoint and streams back a result string.

The inputs and outputs that matter

For a beginner, ignore most of the sampling knobs at first and focus on three things:

• model / custom_model - pick from the dropdown (LLaVA variants, MiniCPM-V, Moondream, etc.) or type any model name from Ollama's library into custom_model if you've pulled something not listed.
• prompt and system_context - both ship with sane defaults ("Describe the following image in detail…"), but this is the field to rewrite if you want tags instead of prose, or a specific focus (clothing, lighting, composition).
• images (IMAGE) is the only wired input; everything else is a widget.

Everything below that - temperature, top_k, top_p, repeat_penalty, seed_number, num_ctx, max_tokens - is standard LLM sampling. Temperature near 0 gives you literal, repeatable descriptions; push it up if you want more creative phrasing. keep_model_alive controls how long Ollama keeps the model loaded in VRAM after the call (-1 = indefinitely, useful if you're chaining several describer calls back to back so it doesn't reload every time). There's also an optional structured_output_format - feed it a JSON schema or Python dict and Ollama will constrain its answer to match that shape, which pairs nicely with the JSON Property Extractor node downstream if you want a specific field out of a structured response rather than a paragraph.

Output is a single result STRING - wire it into a CLIPTextEncode, a Save Text node, or the Text Transformer if you want to massage it further.

How to install it

Two prerequisites before the node itself: install Ollama and pull at least one vision-capable model (ollama pull llava). Then get the node pack via ComfyUI Manager - search "ComfyUI-Ollama-Describer" and install - or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/alisson-anjos/ComfyUI-Ollama-Describer.git
pip install -r requirements.txt

Windows users can run install.bat instead of the pip line. Restart ComfyUI after either path.

Common issues & troubleshooting

The single most common failure, confirmed by how often it shows up in ComfyUI's own logs on similar Ollama nodes, is a connection error like Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it on port 11434 - that's just Ollama not running. Start ollama serve (or the desktop app) first. Second most common: you selected a model in the dropdown that you haven't actually pulled yet - Ollama will error out asking you to ollama pull it. If api_host was changed for a remote or Dockerized Ollama instance, double-check it's reachable and not blocked by a firewall - the default only works for Ollama running on the same machine as ComfyUI.