APNext OllamaVision

Caption an image locally, for free, with nothing leaving your machine - that's the pitch. OllamaVisionNode (APNext OllamaVision) sends your images to a multimodal model running in your own Ollama instance (LLaVA by default) and returns a text description you feed into a generation. It's the local, no-key answer to this pack's GPT/Grok/Groq vision nodes, and it's genuinely handy for bulk work like captioning a LoRA dataset, where paying a cloud provider per image would sting.

The whole thing rests on one prerequisite: Ollama has to be running with a vision model pulled. This node is a client, not a model - no server, no output.

How it works

Ollama serves a multimodal model on an HTTP endpoint; this node POSTs your image and a captioning instruction to it and returns the result. Your hardware runs the model, so quality tracks whatever you pulled.

The inputs worth knowing:

• images (IMAGE) - what to describe.
• custom_model (default llava-llama3:latest) - the Ollama vision model tag. Must be pulled already, and must actually support images (llava, bakllava, llama3.2-vision, etc.). A text-only model here just fails.
• ollama_url (default http://localhost:11434/api/generate) - your Ollama server.
• custom_prompt / additive_prompt - the base instruction, and extra text appended to it.
• fade_percentage (default 15) - blend strength when you pass multiple images.

Then there's the LoRA-captioning feature: flip dynamic_prompt on and the node substitutes ##TAG##, ##SEX##, ##PRONOUNS## and ##WORDS## in the instruction with the tag (default ohwx man - the classic Dreambooth trigger), sex, pronouns and words inputs. That's how you get captions like "ohwx man standing…, using him/his pronouns, ~100 words" for training data, without editing the prompt each time.

Three outputs: output (the full caption STRING), clip_l (a shorter version aimed at a CLIP-L encoder, useful for Flux-style dual-encoder setups), and faded_image (the blended composite of your inputs).

Installing it

ComfyUI Manager: search comfyui_dagthomas, install, restart - or clone and pip install -r requirements.txt as with the rest of the pack. Then, separately, install Ollama and pull a vision model:

ollama pull llava-llama3

No API key needed - just a reachable Ollama with a multimodal model.

Common issues

Connection refused / hangs. Ollama isn't up or ollama_url is wrong. Confirm the server is running and reachable before anything else - this is the usual culprit.

"Model does not support images." You pointed custom_model at a text model. Use a vision one (llava, bakllava, llama3.2-vision) and make sure it's pulled.

Dynamic prompt did nothing. The substitution only fires when dynamic_prompt is on and your instruction actually contains the ##TAG##-style placeholders. If you overrode custom_prompt with plain text and no placeholders, there's nothing to substitute.

Cloud/managed ComfyUI. localhost:11434 is the server's localhost, not yours, and there's rarely an Ollama there. This node is built for a machine you control; on serverless, a GPU-loaded local model (MiniCPM, Phi) or a cloud vision node is the realistic option.