Ollama Image/Text to Prompt

You know the feeling: you see a reference image you love and you cannot describe it well enough to reproduce. The old answer was a WD14 tagger or a Claude screenshot. The local answer is this node - a ComfyUI wrapper around Ollama that hands your image to a vision-language model and gets back a proper prompt, or hands it your keywords and expands them into one. No API key, no cloud account, nothing leaves your machine. If you've already got Ollama installed, this is the easiest way to add "interrogate this image" to a workflow.

The node is one file with a lot of personality under the hood. It calls Ollama's /api/generate endpoint directly over HTTP - you give it an ollama_url (default http://localhost:11434), a model, and a mode, and it does the rest. Your image tensor gets converted to a base64 PNG and sent along with the request. No image connected? Then it falls back to text mode, expanding whatever's in keywords. That's the whole trick: one node, two personalities.

The modes are the real feature

Ten presets live in the pack's presets.py, each a hand-tuned system prompt aimed at a different generation model. You mostly need to know these:

• prompt - dense natural-language prose, the right fuel for Flux or Qwen-image models.
• danbooru_tags - comma-separated Danbooru-style tags with quality prefixes, escaped character names, and weighting. This is the one for Illustrious or NoobAI, which were trained on that exact vocabulary.
• anima and expand_anima - Anima-style prompts (the masterpiece, best quality, score_9... header, character-name-first, artist tags). Anima's newer Qwen text encoder understands tags and language, but these presets match the checkpoint's native format.
• expand_prompt / expand_tags - text-mode only; turn a few keywords into a full prompt or a tag list. Great for unblocking a blank canvas.
• video_prompt / video_storyboard - cinematic, motion-first descriptions aimed at Wan and friends.
• json_extract - structured output, with the JSON fence stripped for you.

There's also custom_prompt, which nukes the preset and uses your own system prompt instead. If you're tired of the author's framing, that's your escape hatch - and it disables the tag-cleaning post-processing.

The inputs that actually matter

• model - a fixed dropdown of six options (qwen3.5:9b default, qwen3-vl:8b, gemma4:e4b, and small variants). Trap: those tags have to exist in your Ollama library. Pull one first with ollama pull, and if the default tag isn't found, you'll get an error string in your output rather than a prompt.
• keywords - in vision mode this appends instructions/rules to the system prompt ("describe her outfit in detail"); in text mode it's the seed text you're expanding.
• seed - passed to Ollama, and the node's IS_CHANGED hook re-runs when it changes. Same seed, same output.
• keep_alive - minutes to keep the model loaded in VRAM; -1 caches it forever. Default 0 means the model unloads after every call, so the first run of each workflow eats the full load time. Set -1 if you're iterating.
• thinking_mode - on for reasoning-capable models, and it surfaces the model's <think> trace on the second output.

Outputs are two STRING lists: text and thought_process. They're lists because a batch of images produces one result per image - wire text into your CLIP Text Encode and you're done.

Install

Two steps, and step zero is having Ollama itself:

# step 0 - Ollama + a vision model (if your library lacks the default tag)
ollama pull qwen3-vl:8b

# step 1 - the node
cd ComfyUI/custom_nodes/
git clone https://github.com/jluo-github/comfyui-ollama-image-to-prompt.git
cd comfyui-ollama-image-to-prompt
pip install -r requirements.txt

Restart ComfyUI and it appears under the Ollama category as "Ollama Image/Text to Prompt". ComfyUI Manager also has it - search the pack title. Dependencies are a mercy: just requests, numpy, and Pillow. No PyTorch download, no CUDA wheels. The heavy part is the model, which is a few GB in your Ollama cache, not in ComfyUI.

Where people get burned

The classic failure is Ollama not running or the model not pulled - the node doesn't raise an error, it returns "Error: ..." as the text output, so the first thing to check when your prompt is a string of text is your terminal. That " BREAK " you see tacked onto every non-JSON output is intentional (the pack appends it, probably for downstream chaining) - harmless, just don't panic. And if a run feels slow, it's almost always keep_alive: 0 reloading the model. Bump it, and image-to-prompt becomes the fast part of your workflow instead of the bottleneck.