Ovis-U1 Image to Text

Captioning, visual question answering, alt-text, describing what's actually in a frame - that's what this node does, and honestly it's the most quietly useful one in the pack. The generation and editing nodes are the showpieces, but any time you have a pile of images and want words for them, this is what you reach for. It's also the node that makes the pack's pitch concrete: the same 3B model that generates and edits images reads them back to you. One model, three jobs, no API calls, no keys.

If you train LoRAs, this is the node you'll actually live in. Captioning your training set is the chore that decides how good the LoRA is - garbage captions, garbage training - and a local, free, private captioner beats paying for an API call per image. It's greedy-decoded and deterministic, so your captions won't drift between runs the way sampled ones do. That's a feature for dataset work.

How it works

The node takes your image and your prompt, wraps the prompt as "<image>\n{your prompt}", builds the model inputs (resizing the image through the visual tokenizer's smart-resize), and calls the model's standard generate(). There's no sampler math to tune here - it's the text half of the model doing what language models do, capped at max_new_tokens and decoded with special tokens stripped. The heavy lifting is all in how the model was trained.

The inputs that matter

• image - a normal ComfyUI IMAGE tensor. Feed it from Load Image, or from anything upstream that produces an image (an upscaler, a detailer, a previous generation). Wire up a Load Image → this node → Show Text chain and you have a captioning station.
• prompt - default What do you see in this image?. This is your instruction, and it's flexible. Want training captions? Try Describe this image in detail for training a diffusion model. Want a one-line summary? Ask for one. Because it's a full language model behind it, the prompt genuinely shapes the output - the usual captioning nodes don't give you this.
• max_new_tokens - default 4096, range 1–8192. That default is generous - it's sized for long multi-turn answers, not captions. For dataset captioning, dropping it to a few hundred makes runs snappier.

The output is a STRING - that's plain text, not an image. The example workflow shows the honest way to use it: wire it into a Show Text node (the pysssss one) so you can actually read the result, or into any text-saving/processing node you like. A bare STRING output won't save itself anywhere, so if your caption goes into the void, that's why - you need to pipe it somewhere.

Installing it

Part of the ComfyUI-Ovis-U1 pack, so it's the pack install:

cd ComfyUI/custom_nodes
git clone https://github.com/neverbiasu/ComfyUI-Ovis-U1.git
cd ComfyUI-Ovis-U1
pip install -r requirements.txt

or grab "Ovis-U1" from ComfyUI Manager. You still need the Ovis-U1 Model Loader in the graph - this node's model input takes the loader's OVIS_MODEL output. First run downloads ~6 GB of weights into models/ovis/.

Common issues & honest expectations

• Slow first token on low VRAM. The 3B model runs fine on a 12 GB card; on 8 GB it offloads and gets sluggish. If you're only doing captioning, you can live with float16 and even CPU - it's text generation, the waits are tolerable.
• It's only as smart as a 3B model. It won't out-caption a big Qwen-VL or a paid API on subtle details like brand names or fine print. For everyday description and structured scene understanding it's solid.
• Deterministic by design. No temperature knob - the node decodes greedily. Good for consistency, mildly annoying if you wanted variety.
• Prompt still matters. Default is fine for a first test; for training data, put real effort into the instruction. The caption you get is only as good as the question you ask.

If you already run this pack for generation or editing, adding a captioning pass costs you nothing - it's the same loaded model. That's the whole point of a unified model, and this node is where it pays off.