👁️‍🗨️ NVIDIA Vision Node 👁️‍🗨️

Image in, text out. The NVIDIA Vision Node sends whatever image you wire into it up to Llama 3.2's vision models on NVIDIA's hosted NIM API and returns a description. Use it for the three jobs people actually wire VLMs into: captioning a LoRA training set, seeding an img2img or image-to-video prompt from a reference image, or just asking "what's in this render?" of your own output.

Worth being clear about where it sits. Captioning is one of the standard LLM-in-the-graph jobs, and the community defaults are local - Florence-2 for speed, JoyCaption for richer language, WD14 for booru tags. This node is the API alternative: you swap the uncensored-offline-free trio for bigger-model quality without hosting anything. The default is Llama 3.2 11B vision-instruct, with 90B variants in the dropdown if you want more. Fine for SFW captioning and prompt seeding. If you need offline or uncensored, reach for a local captioner instead.

How it works, straight from the source. The node takes the IMAGE tensor, converts it to a PIL image, downscales anything over 1024px (LANCZOS), re-encodes to JPEG at quality 90, base64s it, and posts a data-URI image_url to integrate.api.nvidia.com/v1/chat/completions. Non-streaming gets a plain JSON response; flip stream on and it parses the SSE. There's a 1.5-second rate limit between calls, exponential-backoff retries, and an in-memory cache keyed on model + prompts + the md5 of the image bytes - so re-running the same image costs nothing.

Inputs that matter:

• api_key - masked, starts empty.
• model - 4 choices, all Llama 3.2 vision. The -instruct variants are fine-tuned for instruction following; 90B is noticeably slower and pricier.
• prompt - your instruction; the default is "Describe this image in detail…".
• system_prompt, max_tokens (16–4096), temperature, top_p, use_cache - the usual knobs.
• image - the optional IMAGE socket. Load Image into it.

One quirk worth knowing: the default system_prompt is a copy of this pack's text node persona - a "master visual prompt engineer" tuned to emit a ready-to-use image prompt under 300 words. Great if you want an image→prompt assistant. If you want a neutral caption for training data, swap it for something like "Describe this image factually and in detail."

The output is response, a STRING. It feeds a CLIP Text Encode, a Save Text node, or any downstream text consumer. For image-to-video that's the classic captioner→sampler chain.

Install is the same pack either way - ComfyUI Manager (search "ComfyUI-NvidiaVision"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/theshubzworld/ComfyUI-NvidiaVision
cd ComfyUI-NvidiaVision
python_embeded\python.exe -m pip install -r requirements.txt   # only openai is declared

Restart, paste your NVIDIA API key from the developer portal, done.

Gotchas. Without an image wired in, the node still runs text-only - which is pointless, the image is the whole point. Remember the 1024px downscale: fine detail in a big render is gone before the model sees it, so don't expect it to read small text or count eyelashes. The 90B models answer slower than you'd like. And the model list is a snapshot of NVIDIA's catalog - a retired model returns errors until you pick a live one. Same security note as its sibling: it's an API-wrapper node from the category that once shipped malware, it carries your key in the workflow file, and this pack is small and open enough to read in five minutes. Do that before first run.