LLaVA-OneVision Run

This is the node that actually does something. LLaVA-OneVision Run takes the LLAVAMODEL from the loader, an image, and a prompt, and returns one big STRING - the model's answer. The whole pack is basically this node plus two helpers: OneVision Caption Folder is just this called in a loop, and everything else exists to move text around. If you want to ask "describe this image" or "what text is on this sign" inside a ComfyUI graph, this is your workhorse.

How it works, roughly. The node converts your IMAGE tensor to PIL images, pushes them through the model's vision processor, then builds a Qwen chat prompt with the image token stuck on the front of your prompt automatically - so you never add the image placeholder yourself. Then it runs model.generate() with your max_tokens, temperature, and seed. Output is decoded and returned as a plain string. Feed it a batch of images and, on the -ov (one-vision) model variants, it can actually reason across all of them - same call, multiple frames. That's the OneVision trick: it was trained on multi-image and video-frame inputs, not just single stills.

The inputs you'll actually set:

• llava_model - the output of the (Down)Load LLaVA-OneVision Model node. No alternative; they come as a pair.
• image - an IMAGE. Single image or a batch.
• prompt - multiline, your instructions. This is where you get all the leverage. "Caption this in detail" and "What colors dominate?" are both fair game.
• max_tokens - default 4096, up to 8192. Generous, and appropriate: a 7B model writing a detailed natural-language caption eats tokens fast.
• temperature - default 0. At zero the node decodes greedily (deterministic-ish given the seed); nudge it up to 0.2-0.4 if you want variation. The seed is hashed before being applied, so don't expect a literal seed-to-seed mapping, but you do get reproducibility per prompt.
• keep_model_loaded - default true. Flip it to false and the model offloads to system RAM and clears the VRAM cache after each run. Useful when you caption occasionally and don't want a 7B parked in VRAM between queues; the cost is a slow reload next time.

The output is a STRING named result. Wire it to a text display node, a text-saver from another pack, or feed it into SaveCaptionToTextFile (with caveats - see that article) so it lives somewhere visible.

Where people get burned. The most common failure isn't this node, it's the model you picked upstream. A 7B in fp16 wants something like 16GB of VRAM just to sit in; if you're on a 8-12GB card, go back to the loader and pick the 0.5B variant or nf4 quantization. Related: if the loader was set to flash_attention_2 but you don't have flash-attn installed, this node is where the crash surfaces - switch the loader back to sdpa. And don't bother with CPU for the 7B; the loader itself refuses fp16/bf16 on CPU, and a full-size vision model on CPU is a patience test, not a workflow. Install is the pack-wide story: ComfyUI Manager → search "ComfyUI Llava-OneVision", or git clone https://github.com/kijai/ComfyUI-LLaVA-OneVision into custom_nodes, then restart.

One honest note: kijai's README calls this pack a work in progress and the repo's been quiet for a while. It works, but don't expect an actively maintained roadmap. For the core job - a local, private vision-language answer inside ComfyUI - it's still one of the simplest setups you'll find, and the 0.5B model makes it genuinely usable on mid-range hardware.