ialhabbal VLLM

Drop an image in, get a written description out - that's the whole trick. ialhabbal_VLLM runs a Qwen3-VL vision-language model inside ComfyUI, fully local, no API calls, no key. Feed it a generated frame and it hands back a caption you can pipe straight into a CLIPTextEncode, a LoRA trainer, or another LLM. It's the "describe the image so I can re-prompt it" job that used to mean copy-pasting into a browser tab, now sitting in your graph like any other node.

It's part of the ialhabbal_VLLM family in the ComfyUI-ialhabbal suite, which also covers GGUF-quantized models and prompt enhancement. If your only goal is "look at this picture and tell me what it is," this is the node you want.

How it works

The node loads a HuggingFace checkpoint with transformers - specifically AutoModelForVision2Seq plus the Qwen processor. First run calls snapshot_download and pulls the weights into ComfyUI/models/LLM/Qwen-VL, so budget for a multi-gigabyte download and some disk space before you hit run. The model stays loaded between runs if you leave keep_model_loaded on, which is the default and the right choice unless you need the VRAM back.

Images go in as a tensor; the processor turns them into tokens the model can see. Video inputs are handled by sampling frames - the base node pulls 16 evenly-spaced frames, so a 10-second clip doesn't blow up your context.

The three inputs a beginner actually touches:

• model_name - 18 Qwen3-VL checkpoints, from 2B up to 32B, including Thinking variants and FP8 quantized releases. The default Qwen3-VL-2B-Instruct is a fine place to start.
• quantization - None (FP16) for best quality, 8-bit for 8–16 GB cards, 4-bit for 6 GB or less. The tooltip is blunt about the trade: 4-bit fits small VRAM but runs slower.
• preset_prompt - the instruction that tells Qwen what to do. Options range from Tags and Simple Description through Cinematic Description, Video Summary, and even Short Story. Fill custom_prompt and it completely replaces the preset.

max_tokens caps answer length (default 512), seed controls both sampling and which video frames get picked. Leave attention_mode on auto - it tries flash-attn v2 when installed and falls back to the built-in SDPA otherwise.

The single output is RESPONSE, a plain STRING. Wire it into a CLIPTextEncode text input, a Show Text node, or anywhere else a string goes.

Installation

Install the whole suite once - the VLLM nodes ride along:

cd ComfyUI/custom_nodes
git clone https://github.com/ialhabbal/ComfyUI-ialhabbal.git

Then restart ComfyUI. Or search ComfyUI-ialhabbal in ComfyUI Manager and hit Install. The VLLM nodes need the heavy transitive dependencies from the pack's pyproject (transformers, huggingface-hub, bitsandbytes, accelerate, hf_xet, and llama-cpp-python[server] for the GGUF siblings), so the first launch after install may take a while as pip sorts that out.

Common issues

• First run looks frozen - it's downloading weights. Check the console; snapshot_download can sit there for a while on a big checkpoint.
• OOM on load - step down a quantization level or pick a smaller model. A 32B at FP16 is a 28 GB affair; that's not the default for a reason.
• Flash-attn errors - unnecessary. auto handles the fallback; only override attention_mode when you're debugging a specific backend.
• VRAM never comes back - keep_model_loaded is on. Flip it off when you need the card for the diffusion side, and accept the reload cost per run.

It's a captioner, not a fortune teller. Like every VLM, Qwen3-VL will occasionally mix up which person is wearing what in a busy frame - read its output before you treat it as ground truth.