🦙👁 Ollama Vision

This is the local, free alternative to the cloud vision-API nodes floating around ComfyUI - feed it an image, it hands the image to a vision-capable Ollama model running on your own machine, and you get a text description back. No API key, no per-call bill, no internet dependency once the model's downloaded. The trade-off is the usual local-model one: quality depends entirely on which vision model you've pulled, and it's only as fast as your own GPU or CPU.

How it works

Point it at an image and give it a prompt (the OLLAMA_VISION_PROMPT input) describing what you want back - a caption, a list of objects, a semantic description, whatever. By default this uses moondream, a small, fast vision model, but you can swap in anything you've pulled locally through the pack's Ollama Configuration node. The pack also ships a dedicated Ollama Vision Prompt Selector node with a menu of canned prompt templates (basic caption, character description, object list, prompts formatted for SDXL or FLUX, and more) built to plug straight into this node's prompt input instead of you writing one from scratch every time.

vram_retention_minutes is worth understanding before you turn it on: it keeps the model resident in VRAM after the node finishes, so a second call reusing the same model is significantly faster. The trade-off, straight from the author: that VRAM stays occupied the whole retention window, which is a bad idea if you're also trying to run an image generation in the same workflow - don't combine "keep vision model warm for ten minutes" with a heavy checkpoint load right after it unless you've got VRAM to spare.

The inputs and outputs that matter

• IMAGE - the image to describe.
• OLLAMA_VISION_PROMPT - the instruction text for what kind of description you want; wire this from the pack's Vision Prompt Selector node, or just type your own.
• vram_retention_minutes (0–99, default 0) - how long to keep the model loaded in VRAM after this run. Zero unloads immediately.
• seed (default −1) - passed through to the model call; leave at −1 for a fresh result each run.
• answer_single_line - forces the response onto one line, which matters if you're piping the output straight into another node that chokes on line breaks (a prompt field, for instance).
• Optional: OLLAMA_CONFIG (from the Ollama Configuration node, to override the default model/URL) and context (extra text context for the request).
• Output: response - the model's text description.

How to install it

Through ComfyUI Manager: search Bjornulf_custom_nodes, install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/justUmen/Bjornulf_custom_nodes

cd in, pip install -r requirements.txt (venv recommended), restart ComfyUI. You'll also need ollama pull moondream (or whatever vision model you plan to use) done separately through Ollama itself before this node has anything to talk to - installing the node doesn't install a model.

Common issues

"Connection refused" almost always means either Ollama isn't running, or the default http://0.0.0.0:11434 in the node's config doesn't actually reach it - common if ComfyUI is running in a container or on a remote box. Point an Ollama Configuration node at the real address and it clears up.

A "model not found" style error means moondream (or whatever model you configured) hasn't actually been pulled locally yet - ollama list will tell you what you've got. And if descriptions come back oddly generic or off-target, that's usually the prompt, not the node: moondream is small and fast but not sophisticated, so swapping to a larger vision model via Ollama Configuration is the first thing to try before assuming something's broken.