🟨 NVIDIA Captioner

You're assembling a LoRA dataset and your GPU is already at 100% - or you're on a machine that can barely run the checkpoint, let alone a vision model. Normally that's a wall: captioning tools like JoyCaption and Florence-2 run locally and eat VRAM on top of everything else. NVIDIA Captioner takes the other road. It ships your images to NVIDIA's hosted inference API, gets captions back from a Llama 3.2 vision model, and writes a .txt file next to each image in the exact format ai-toolkit and Kohya expect. No local vision model, no extra VRAM, just a network request per image.

This is a niche but genuinely useful move. The community's advice on captioning is clear - quality matters more than dataset size, and the gold standard is still doing it by hand. But when you have hundreds of reference images and no headroom, a cloud model beats not captioning at all. Just remember the trade: each image is a paid API call, and your pixels leave your machine. Fine for reference shots of clothes and furniture; think hard before sending 500 personal photos through it.

How it works

The node doesn't take a ComfyUI image input. It takes a filesystem path (image_directory) and reads every .png, .jpg, .jpeg, .webp, .bmp, and .tiff in the folder itself. Each image is downscaled to at most 1200px, converted to JPEG, base64-encoded, and posted to NVIDIA's OpenAI-compatible endpoint at integrate.api.nvidia.com/v1/chat/completions with a Bearer API key. The response is written to <same-name>.txt beside the image, which is the standard LoRA dataset layout.

Two things in the code are worth knowing. First, it rate-limits itself to one request every 1.2 seconds and retries failures with exponential backoff - that's a throttle, not a speed setting, so don't expect blazing throughput. Second, there's an in-memory MD5 cache keyed on model + prompt + image hash, so re-running a batch with use_cache on won't burn credits on images you already captioned. The cache dies on restart, though.

It also reads NVIDIA_API_KEY from your environment or a .env file (the author uses python-dotenv), so you don't have to paste your key into the node's masked password field every time.

The inputs that matter

• image_directory - the folder to caption. This is a path string, so use an absolute path and double-check it, because there's no file picker.
• api_key - from build.nvidia.com, where you sign up for a free account and get credits for the hosted NIM API.
• model - defaults to meta/llama-3.2-11b-vision-instruct; the 90b variants are smarter, slower, and pricier. Start with 11b.
• system_prompt_preset - the author ships five tuned prompts: default, women_influencer, cartoon_character, interior_design, product_shot. The last four are clearly aimed at niche LoRA datasets (the source even references WAN 2.2 LoRA prompts). If none fit, custom_system_prompt overrides the preset entirely, which is where you'd paste your own "describe the clothing, not the background" instructions.
• prompt - the user message sent with each image, defaulting to "Describe this image in detail."
• skip_existing_txt - on by default; images that already have a .txt are skipped and their existing caption is read instead. Leave it on for resumable runs. Turn it off when you change a prompt or preset and want everything re-captioned, or your old captions will silently survive.

The rest - max_tokens, temperature, top_p, penalties, retry knobs - pass straight through to the API. The defaults are fine; drop temperature toward 0.2 if you want more reproducible captions.

Outputs are all_captions (a filename -> caption log you can pipe to a text node) and last_caption. But the real output is the .txt files on disk - that's what feeds your trainer.

Installation

Install via ComfyUI Manager (search "ComfyUI-NvidiaCaptioner") or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/theshubzworld/ComfyUI-NvidiaCaptioner.git

Then restart ComfyUI. One thing the README gets wrong: it tells you to pip install -r requirements.txt, but no such file ships in the repo. It doesn't matter - the only dependencies are requests, Pillow, and python-dotenv, all of which come with ComfyUI already. There are no model downloads because there's no local model.

Where people get burned

Most issues are path or key problems: an empty or wrong image_directory returns "invalid directory", and no key (node or env var) returns an immediate error. Network hiccups surface as retry logs and, eventually, an exception. And because the node is new and small - zero real-world signal on the usual forums - treat it as "works as written," not "battle-tested." If a run looks wrong, toggle skip_existing_txt off, nuke the stale .txt files, and rerun. It's a thin, honest node that does one cloud API call per image well; the failure modes are mostly the ones you bring to it.