InternVL HF Inference

This is the node that actually answers

The other two nodes in this pack set the table. InternVL HF Inference is the one that reads the image, thinks about it, and hands you text - a caption, a description, an answer to "what's wrong with this render?", whatever you prompt it to produce. It's the payoff node, and it's where you spend your time tuning prompts instead of wiring.

InternVL2 was the go-to open captioning model back when this pack shipped, and it's still genuinely good at it - community comparisons of the 8B put it comfortably ahead of Llava-class models, trading blows with the then-new JoyCaption on FLUX-era caption quality. For "describe this image well," it holds up today, even if the model family has moved on to newer generations.

How it works

The name means what it says: this is native transformers inference. Unlike the pack's (unregistered, commented-out) LMDeploy node, this one calls the model's own chat() method directly. The source is short and readable:

• It builds the question as <image>\n{system_prompt}\n{prompt} - image token first, then your system prompt, then your question. InternVL's chat format.
• A generation_config gets assembled from num_beams, max_new_tokens, and do_sample, and passed to chat() along with the tokenizer and the preprocessed image tensor.
• After generation, unless you told it not to, the model is offloaded to CPU and the GPU cache is cleared - so it doesn't sit in your VRAM between runs.

The image input expects the normalized pixel-values tensor that Dynamic Preprocess outputs, not a raw ComfyUI image. Feed it a plain image and the answers come back unhinged, because InternVL2's chat format wants those normalized tiles. The full happy path is: Model Loader → Dynamic Preprocess → this node → text out.

Inputs and outputs

Required: image (from Dynamic Preprocess), model (the InternVLModel from the loader), system_prompt (default "You are a helpful assistant."), and prompt (default "What is this?"). For captioning, that system_prompt does a lot of work - community experience with InternVL2 is unanimous that changing the system prompt changes the output more than almost any other knob, so treat it as your main tuning surface.

The optional group is where the practical decisions live:

• keep_model_loaded (default off) - the model unloads after every run, so a batch of images pays the offload/reload tax each time. Flip this on for batch captioning if your card can hold the model; it pins the VRAM until ComfyUI frees it.
• max_new_tokens (default 1024, up to 4096) - the cap on reply length. Generous for captions; you'll rarely hit it.
• do_sample (default off) and num_beams (default 1) - off + 1 beam is greedy decoding: deterministic, literal, a bit flat. If your captions all sound the same and you want variety, switch do_sample on and set num_beams to 2–3.

The single output, output, is a plain STRING. Wire it into a text-display node or Save Text to keep your captions. That's the whole node - no images out, no branching. You ask, it answers.

Install

It's part of the ComfyUI_InternVL2 pack, same as the other two. ComfyUI Manager (search "InternVL") or:

cd ComfyUI/custom_nodes
git clone https://github.com/leeguandong/ComfyUI_InternVL2

Then restart. First run downloads the model from Hugging Face (see the loader article for the size math), and remember this pack has no requirements.txt - if transformers isn't in your environment you'll need pip install transformers huggingface_hub.

Common issues

• Nonsense answers → you're feeding a raw image instead of the Dynamic Preprocess output. Fix the wire.
• Slow batch runs → the default keep_model_loaded=False is offloading the model between every image. For captioning a folder, turn it on.
• Stalled-looking first run → that's the model download; watch the console, not the UI.
• Flat, samey captions → you're in greedy mode. do_sample on, or raise beams.