QwenVL-F

The name is a little lie: despite the "F," nothing here calls an API, and there's no key to paste. QwenVL-F runs Alibaba's Qwen3-VL (or Qwen2.5-VL) vision-language models locally inside your ComfyUI graph, so it can describe an image, summarize a video, or read a picture and turn it into a prompt - all on your own GPU. This is the standard Transformers-backed node from the pack, the one to reach for first.

Why bother wiring a VLM into a diffusion workflow at all? Because it does jobs that no diffusion node can. Feed it a reference image and it writes a caption for your img2img or image-to-video prompt. Point it at a folder of images and it produces the natural-language captions for a LoRA training set (the pattern llm-in-comfyui.md calls "VLM captioning"). It's an understanding model, not a generator - it reads, it doesn't render.

The fork's one rule: models you download yourself

This is a fork of 1038lab's popular ComfyUI-QwenVL, and the fork's whole personality is "no automatic downloading." Upstream grabs models from Hugging Face on first use; this pack removes every snapshot_download and hf_hub_download call. The model dropdown is built by scanning your disk - ComfyUI/models/text_encoders and ComfyUI/models/LLM, recursively, plus anything your extra_model_paths.yaml points at. You place the model folder, reload ComfyUI, and it appears. If it's missing, the node raises an error that lists exactly which folders it looked in. It's slightly more work and a lot more predictable.

Under the hood it's standard stuff: transformers loads a AutoModelForVision2Seq checkpoint, bitsandbytes does on-the-fly 4-bit or 8-bit quantization, and the attention backend resolves auto → Sage → Flash Attention → SDPA (FP8 and bits-and-bytes models get forced to SDPA regardless). A video input is treated as a frame sequence; seed controls both sampling and frame picking.

The inputs that matter

• model_name - a dropdown of what's on your disk. If it says (no models found - see console), the scan found nothing.
• quantization - None (FP16) for quality, 8-bit (Balanced) for 8–16 GB cards, 4-bit (VRAM-friendly) for 6 GB or less (slower).
• preset_prompt / custom_prompt - nine built-in jobs (Tags, Detailed Description, Video Summary, Prompt Refine & Expand…) or your own text, which fully replaces the preset.
• max_tokens, enable_thinking, keep_model_loaded, seed - the usual suspects. enable_thinking turns on <think>...</think> reasoning, and only does anything on the Thinking variants of Qwen3-VL.

Optional inputs are image and video (both IMAGE type). The single RESPONSE output is a STRING - wire it into any text encoder node, or a text display, and feed the result into the sampler.

Install

ComfyUI Manager: search "ComfyUI-QwenVL-F". Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/id-fa/ComfyUI-QwenVL-F
pip install -r requirements.txt

Then restart ComfyUI and drop a model folder under models/text_encoders or models/LLM. Two gotchas: Qwen3-VL needs transformers >= 4.57.0, and the Instruct checkpoints carry the usual assistant-style refusals - if you're captioning explicit material, people run abliterated Qwen3-VL builds instead.

Troubleshooting

(no models found - see console) means the scan found nothing - your model folder needs a config.json sitting next to its weight shards, or it isn't a valid HF checkpoint. If you see a model name error, re-read the listed paths: the fork really will not download for you. Slow first run is normal (that's the load + first inference); after that keep_model_loaded makes subsequent runs fast. VRAM tight? Drop quantization to 8-bit or 4-bit.