QWEN VLM MACHINE (GGUF)

The GGUF half of the "QWEN VLM MACHINE" pair exists for one reason: to run a 7B vision-language model on hardware that can't hold the fp16 version next to a diffusion checkpoint. GGUF is llama.cpp's quantized weight format - the Q4_K_M default in this node is a few gigabytes instead of ~16 - and because it runs through llama-cpp-python, it also works on Macs (Metal) and CPU-only boxes, which the transformers-based sibling node can't honestly claim. If you're on a 12 GB card, or you're on a Mac, this is the node you want.

The trade-off is honest and worth stating up front: the GGUF path is smaller and lighter, but it's also a single-frame viewer. Feed it a video and it just takes the middle frame - llama.cpp's vision handler in this pack only handles one image natively. If you need actual video understanding, the Safetensors machine node does that properly.

How it works

Two files, not one. GGUF vision models split the weights into the quantized language model (e.g. Qwen2-VL-7B-Instruct-Q4_K_M.gguf) and a separate vision connector, the mmproj file (e.g. qwen2-vl-7b-vision.gguf). You type both paths into model_path and mmproj_path - plain strings, no file picker - relative to your ComfyUI folder. The README's examples point at models/LLM/..., and you can drop them in models/checkpoints too; the node doesn't care where they live as long as the path resolves.

The node builds a Llama instance with a Llava15ChatHandler wired to the mmproj file, so the image gets base64-encoded as a JPEG data URL and handed to the model through a chat completion. Output is one text STRING. It's cached like the Safetensors node, so the model stays resident between runs unless you switch keep_model_loaded off.

The inputs that matter

• model_path / mmproj_path - the two required files, from HuggingFace. Missing either and you get nothing useful out.
• gpu_layers - default -1 offloads everything to the GPU. On a CPU-only box set it to 0; on Mac, -1 uses Metal.
• ctx - the context window, default 8192. Leave it unless you're feeding long prompts in.
• preset_prompt / custom_prompt - same five presets and free-text override as the Safetensors node.
• max_tokens, temperature, top_p, top_k, repetition_penalty - standard sampling knobs.

The generated-token budget is 384 by default, lower than the Safetensors node's 512 - worth bumping if you're asking for detailed descriptions.

The gotcha people actually hit

The reload key. In the GGUF node, seed is part of the cache check - the same one that watches model_path, mmproj_path, ctx, n_batch, and gpu_layers. What that means in practice: randomize the seed widget and the node tears down and reloads the whole model from disk before running. On a big GGUF that's a multi-second stall every single queue. Batch or iterate with a fixed seed, and only change the seed when you actually want different output.

Installing

Same pack, same steps as the Safetensors node:

cd ComfyUI/custom_nodes
git clone https://github.com/darkpool999/ComfyUI-Qwen2-VL-Nodes
cd ComfyUI-Qwen2-VL-Nodes
pip install -r requirements.txt
pip install "llama-cpp-python>=0.2.75"

llama-cpp-python is deliberately optional in the requirements file, which tells you something: on Windows it wants a working C++ toolchain (MSVC build tools) or a prebuilt wheel, and the author didn't want that bricking installs for people who only use the Safetensors node. If you skip it and run the GGUF node anyway, you don't get a crash - you get an error string in the text output telling you to install it. If it fails to build, grab a prebuilt wheel for your platform instead of fighting the compiler.