🤖 Vision Language (Transformers)

Image captioning is the gateway drug of local vision models: you drop a render in, you get a paragraph of prose back that describes composition, lighting, and whatever it is that's wrong with those hands. This node is the pack's full-fidelity way to do that. It runs a complete Qwen3-VL checkpoint through HuggingFace Transformers - the heavier sibling of the pack's GGUF vision path. Same idea, more machinery, better answers on the questions that actually take reasoning.

How it works

The node needs a model_config of type TRANSFORMERS_MODEL, which comes from the pack's VisionModelLoaderTransformers node. That loader is where the real choices live: it lists ten Qwen3-VL checkpoints - instruct and thinking variants at 4B and 8B, plus FP8 and abliterated builds - and passes along quantization, attention, and a keep_model_loaded flag to the engine.

This node then runs generation. The inputs that matter:

• prompt - default "Describe this image in detail."; the actual question you're asking.
• image - optional single image; the thing being described.
• temperature (0.7), top_p (0.8), top_k (20), repetition_penalty (1.0) - the tooltips quote these as the Qwen3-VL recommended values, so they're already where you'd want them.
• max_tokens - default 2048, but the tooltip recommends up to 16384 for Qwen3-VL when you're asking for long, detailed captions.
• seed - lock it for reproducible answers.
• system_prompt - optional; a good spot for "only describe technical flaws" style framing.

Output is a single context string. The node is an output node, so the answer displays on it directly, and you can wire the text wherever strings go.

Where people get burned

This is the heavyweight path, and it's worth being honest about that. First run downloads the full checkpoint from HuggingFace - several gigabytes, full precision unless you picked an FP8 variant. Once loaded it wants real VRAM. The engine loads the model on demand, so the first generation pays the load cost; keep_model_loaded on the loader keeps it resident between runs, which is what you want if you're running several captions in a row.

If your card is tight, this is exactly the node where the pack's GGUF vision path becomes the smarter pick - the whole reason GGUF quantization exists is to run these models on hardware that can't hold the fp16/fp8 original. If you have the VRAM (or you're on a 4B model), the Transformers path gives you the model as the authors shipped it, and it shows on hard questions.

Install

ComfyUI Manager (search "ComfyUI-GGUF-FX") or:

cd ComfyUI/custom_nodes
git clone https://github.com/weekii/ComfyUI-GGUF-FX.git
cd ComfyUI-GGUF-FX
pip install -r requirements.txt

Restart, then let the first run download the checkpoint while you make tea. And if you hit an out-of-memory error mid-caption, the fixes are the boring ones: drop to a 4B model, take the FP8 build, or cut max_tokens. Change one thing at a time and lock the seed - you're comparing prose now, not pixels, but the debugging rule still applies.