👁️ FMJ Llm Ollama Vision

Image in, words out - that's the whole pitch. 👁️ FMJ Llm Ollama Vision sends an image from your ComfyUI graph to a multimodal model running in Ollama and returns a text description. Sound simple? It is, and that simplicity is exactly what makes it useful, because "image becomes a prompt" is the foundation of a lot of good workflows: alt text for a site, an art-critique pass, or the workflow that matters most - describing a frame so a video model can build a prompt that matches it.

The default model is qwen3-vl:2b, with llava, bakllava, and moondream all viable alternatives. Anything multimodal in your ollama list will do. And to be clear about what it isn't: nothing leaves your machine. The image goes to your local Ollama server as base64, and the description comes back. That's the whole round trip.

How it works

Under the hood it's the vision sibling of the pack's Prompt Generator. The node takes your IMAGE tensor, squeezes off the batch dimension, converts it to a PIL image, encodes it as a base64 PNG, and attaches it to the user message of a chat call. The system instruction comes from the same CSV system the generator uses - except this one reads the csvv/ folder, and the two columns are description_type and system_prompt. Each row becomes an entry in the description_type dropdown. The pack ships seven presets (art_style, lighting, objects, detailed, composition, characters, simple) and you can drop in your own CSVs whenever you want a different analysis angle.

The inputs and outputs

• image (required) - any IMAGE output: Load Image, a KSampler result, whatever.
• description_type - the analysis mode from your csvv/ files.
• model_name - default qwen3-vl:2b; must be installed via ollama list.
• disable_thinking (default ON) - same story as the generator: keeps Qwen3-VL from burning its output on a <think> block and returning nothing. Leave it on.
• override_prompt (optional) - replaces the CSV instruction for one-off questions.
• max_tokens, temperature, seed, keep_alive, request_timeout - the usual suspects.

You get two STRING outputs: description (what the model said) and debug_info (status, model, analysis type - your first stop when description comes back empty).

Where people get burned

The same three traps as the generator, with one extra:

1. Qwen3-VL is slow, and the thinking-mode empty response is real. disable_thinking on, request_timeout high - though fair warning, like the pack's other nodes, request_timeout is reported in debug_info but not actually wired into the HTTP call in this version. If it times out, the model is slow, not the setting.
2. Chat multimodal models only. This uses /api/chat, so a text-only or non-chat model will just fail.
3. The security reflex. "LLM vision node for ComfyUI" is the exact category that produced the infamous ComfyUI_LLMVISION malware incident in 2024 - a malicious node that got installed by people who didn't look at what they were cloning. This pack is benign: it talks only to your local Ollama, the source is short and readable, and you can audit all four nodes in a minute. But the fact that this category earned that reputation is a good reason to get in the habit of skimming a pack's source before you install. Custom nodes are arbitrary Python; that's the ecosystem's deal.

Installing it

Same pack install as always: ComfyUI Manager → search "ComfyUI_FMJ_LLM", or cd ComfyUI/custom_nodes && git clone https://github.com/bulldog68/ComfyUI_FMJ_LLM, restart, and make sure the ollama pip dependency landed. Then pull a vision model: ollama pull qwen3-vl:2b.

The workflow worth building: Load Image → this node describes it → feed that description into the pack's Prompt Generator with the LTX-Video style → video prompt that actually matches your starting frame. Community workflows have been doing exactly this with Qwen-VL models for a while now; this pack just puts both halves of it on your local machine.