QWEN VLM MACHINE (Safetensors)

This is the node to reach for when you want a real vision-language model doing a real job inside your workflow - not an API call, not a tiny captioner, but a full Qwen2-VL 7B reading your image on your own GPU and writing text about it. It's the heavier of the two "QWEN VLM MACHINE" nodes in this pack (the other one loads GGUF quants via llama.cpp, see the sibling page). If you have the VRAM for it, this is the more capable path, and the one that actually understands video, which the GGUF node quietly can't.

Quick context, because the terminology trips people: this isn't the LLM inside a checkpoint that encodes your prompt. It's a separate model you bolt into the graph as a tool. It reads an image and produces text - captioning, describing, or turning an image into a seed prompt for img2img or image-to-video. That's the "LLM in the graph" pattern, and this pack is a local, uncensored, per-call-free implementation of it. No API key anywhere.

How it works

On first run the node downloads the model from HuggingFace using the model_name you type in - the default Qwen/Qwen2-VL-7B-Instruct is roughly 16 GB in fp16, or 5-6 GB once 4-bit kicks in. You pick quantization (4-bit nf4, 8-bit, fp16, or bf16), an attention_mode, and a device (auto/cuda/cpu), and it loads via transformers with a BitsAndBytesConfig. If flash_attention_2 fails because you don't have flash-attn installed, the code catches it and silently falls back to eager - nice touch.

The model is cached in a global, so it stays in memory between runs and only reloads when you change model_name, quantization, attention_mode, compile, or device. That's the behavior to understand for batch work: leave it alone and it's fast; touch quantization and you eat a full reload.

When you wire in a video, it evenly samples down to frame_count frames (default 16), dumps them as temp JPEGs, and feeds them as an actual video input at 1 fps. If you wire both image and video, it gets both. Output is a single text STRING.

The inputs that matter

• model_name - the HuggingFace id. The default 7B is the sensible start; Qwen/Qwen2-VL-2B-Instruct is there if 7B won't fit.
• quantization - 4-bit is the default and the right answer on a 12 GB card that also has to hold a diffusion model.
• preset_prompt / custom_prompt - five presets (Detailed Description, Cinematic, Booru Tags, Short Caption, Custom) plus your own text appended. This is how you steer what the model says.
• frame_count - how many video frames to sample.
• keep_model_loaded - leave it on for speed; turn it off if the VLM and your checkpoint are fighting over VRAM.

max_tokens, temperature, top_p, num_beams, repetition_penalty, and seed behave like you'd expect. One thing you should know: the pack hardcodes an "unfiltered, uncensored" system prompt into every call. For captioning NSFW training data that's a feature, for polite image descriptions it just means less moralizing.

Installing

ComfyUI Manager works (search "Qwen2-VL"), or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/darkpool999/ComfyUI-Qwen2-VL-Nodes
cd ComfyUI-Qwen2-VL-Nodes
pip install -r requirements.txt

Then restart ComfyUI. The requirements pull in transformers, accelerate, bitsandbytes, qwen-vl-utils, and einops - bitsandbytes is what actually powers the 4-bit/8-bit options, so it's non-negotiable if you're using quantized loading. On the portable Windows build, install into the embedded Python (python_embeded\python.exe -m pip install ...) rather than whatever system Python happens to be on PATH.

Troubleshooting

The classic trap: the node's text output is invisible. It isn't an output node, so nothing displays by default - wire it into the pack's Qwen2-VL Text Preview node (or any text display) or you'll think it produced nothing. Second: 4-bit needs a working bitsandbytes; on Windows use a recent version with prebuilt wheels. And remember the first run downloads gigabytes from HuggingFace - on a slow connection it looks hung while it's actually working. Given this is a fresh, little-known pack that downloads models and runs arbitrary Python, it's worth a glance at the source before you trust it with anything sensitive - that whole "LLM captioner" category has a bad security track record.