Ollama Video Analyze

Ollama Video Analyze takes a sequence of frames and has a local vision model tell you what's in them. Feed it the IMAGE batch from a VHS Load Video (or any node that emits a frame sequence) and it samples a handful of frames, sends them to an Ollama vision model, and hands you back a text description. "Describe what happens in this video clip" is the default prompt, and that's exactly the job - automated QC on renders, a caption for an image-to-video prompt, or just an understanding of what your own generated clip actually contains.

The honest use-case for most people: you generate a video, and you want to know if it's garbage without scrubbing through it. Frame sampling plus a VLM is a cheap proxy for "did the subject stay consistent, is the motion sane, did it do the thing I asked." It's the same captioning/VLM pattern the KB documents for stills - this just extends it to time.

How it works

Two halves: sampling, then inference. First it picks frame indices from your batch using the sample_mode:

• uniform - evenly spaced across the whole clip (the default, and usually what you want for "what is this video").
• first - the opening frames only.
• last - the closing frames only. Handy when the interesting thing happens at the end.

max_frames caps how many it sends (1–64, default 8). Each selected frame is rendered to PNG, base64-encoded, and sent via Ollama's images[] field to a vision model with your prompt. There's a think flag for chain-of-thought on models that support it, and a format switch between text and json for structured output.

The inputs that matter

• frames - required IMAGE batch. Any frame sequence works: VHS Load Video is the classic source, but a batch of generated frames is fine too.
• prompt - what you want to know about the clip.
• max_frames - the richness/memory dial. More frames = better understanding, slower, more VRAM and context. 8 is a sane start; don't blindly max it.
• sample_mode - uniform / first / last, as above.
• system (optional) - steer the model's framing, defaults to a "you are analyzing frames sampled from a video clip" persona.
• connectivity (optional) - Ollama Connectivity for server/model; this or meta is required, same as the other chat-style nodes.

Outputs: result (the answer), thinking (chain-of-thought text when enabled), and meta for chaining into another Ollama node.

The model decides the quality

This node has no built-in vision; the model you point it at has to be a VLM. qwen2.5vl (7B/32B) is the strong pick for frame understanding, gemma3 is a lighter option, and a text-only model will confidently describe nothing at all. Pull one with:

ollama pull qwen2.5vl

Install

It's in the ComfyUI-OllamaOmni pack. ComfyUI Manager → search "OllamaOmni", or:

cd ComfyUI/custom_nodes
git clone https://github.com/ckinpdx/ComfyUI-OllamaOmni
pip install -r ComfyUI-OllamaOmni/requirements.txt

Restart, and keep Ollama running. The requirements file is just ollama, so nothing heavy hides in the install.

Where people get burned

• Text model, no vision. Wrong output, wrong cause. Check the model.
• Too many frames. 64 frames of PNG at high resolution is a wall of tokens and VRAM. Resize the frames before they reach this node, and keep max_frames modest - you gain little from 64 frames on a 10-second clip, and you pay for it.
• format = json but the model won't comply. Small VLMs are unreliable at strict JSON; qwen2.5vl is better at it than most.
• Uniform sampling misses the point. If the action is only in the last second, first mode will tell you about the establishing shot. That's what last is for.
• No connectivity. Like the other OllamaOmni chat nodes, it refuses to guess a server - wire up Ollama Connectivity (or chain meta from another node) or it errors immediately.

And the usual custom-node caution applies: it's arbitrary Python on import from a GitHub repo, so prefer the official pack and glance at the source before making it a permanent part of your graph.