Ollama Image To Text

Ollama Image To Text is the local-vision cousin of the pack's Ollama chat node: it takes an IMAGE tensor, base64-encodes it, and hands it to a vision model running in Ollama on your machine. No API key, no cloud, no data leaving your box - which is the whole point if you're captioning reference images you'd rather not upload to a third party.

The default query is a bit of a typo'd artifact - "Describe the main content of the picture under the speed，Do not generate any descriptive text." - which reads like a bad translation and contradicts itself. You'll want to replace it with something like "describe this image in detail as a stable diffusion prompt" or "what is in this image?". The output is a single STRING named description, ready for a text preview or to feed the pack's Prompt Extractor.

Inputs: images (the IMAGE input - note it loops over every frame/batch item, so a batch yields one description per image), query, model (populated from your installed Ollama models), seed, top_k, max_tokens, and keep_alive (boolean here - true keeps the model warm for 5 minutes, false unloads it after the call, which matters if you're juggling VRAM).

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/yiwangsimple/ComfyUI_GroqChat

Then install the missing ollama Python package (it's absent from requirements.txt):

pip install ollama

And you need a vision-capable model pulled and Ollama running: ollama pull llava (or bakllava, or a recent llama3.2-vision).

The two gotchas

First, the same import-time trap as OllamaTextToText: the model dropdown is fetched once when ComfyUI starts. If Ollama wasn't up, you get "No models found" and have to restart ComfyUI after starting Ollama. Second, the vision model must actually be a vision model - a text-only LLM will error or hallucinate. The pack's README links the project's inspiration sources, and the authors were explicit that you need a multimodal model.

Honest take: compared to this pack's other vision options, moondream2 (dwimage2) is smaller and dumber, while the Gemini Vision node is smarter but cloud-only. This node is the middle ground: local and private, as smart as the model you pulled. For a privacy-conscious captioning pipeline it's the one you want; just expect quality to track your model choice.