Qwen 3.5 (GGUF)

Qwen 3.5 is a natively multimodal reasoning model - it looks at an image and talks back, and this pack lets you drop that into a ComfyUI graph. The GGUF node is the version you actually want for throughput: the README claims ~152 tokens/sec against ~17 for the transformers sibling on the same RTX PRO 6000. Nine times faster is the difference between a caption that lands in ten seconds and one you wait a minute for. If you're captioning a dataset or analyzing a pile of reference images, this is the one.

But here's the catch, and it's the whole game: this node doesn't use transformers at all. It shells out to llama.cpp's multimodal CLI, llama-mtmd-cli, as a subprocess, feeding it the GGUF weights, the vision projector, your prompt, and your image. That binary is not something pip install puts on your machine - llama-cpp-python does not include it, despite what you'd expect. You build llama.cpp from source, or the node fires back a long, apologetic "llama-mtmd-cli not found" error and tells you exactly how to fix it.

Each model lives in ComfyUI/models/LLM/<model-name>-GGUF/ and needs two files: the GGUF weights and mmproj-BF16.gguf, the vision projector that lets a text model actually see. First use auto-downloads both from unsloth's HuggingFace repos; you can also place them manually and skip the download entirely.

The inputs that matter

• model - 0.8B to 27B, default 9B. On Q4 that's roughly 1GB to 17GB, so the dropdown maps cleanly onto your VRAM.
• quantization - 17 choices, default Q4_K_XL. The "XL" quants are Unsloth Dynamic, which mixes precision per-layer instead of smearing one quant over everything - smarter than the classic Q4_K_M at the same ballpark size.
• n_gpu_layers - 99 offloads everything (default), 0 is CPU-only.
• enable_thinking - off by default here, unlike the transformers node. Flip it on if you want chain-of-thought, and remember thinking eats tokens before the actual answer.

Both outputs are strings: RESPONSE (thinking stripped) and THINKING (the reasoning, empty if disabled). Wire RESPONSE into any text display node - the shipped workflow uses PreviewAny - or string it into a prompt-rewriter loop.

How fast, and at what cost

The GGUF quality ladder is the familiar one from diffusion: Q8 is essentially fp16 at half the size, Q5 is the last tier where you can't see the difference, and Q4 is the accepted compromise when VRAM is tight. On this node the sweet spot is Q5_K_XL or Q8_0 if they fit, Q4_K_XL if they don't - at 6GB, 6.7GB, and 9.5GB respectively on the 9B. One thing it can't do: no video input. Video understanding is the transformers node's job.

Installing (the real part)

cd ComfyUI/custom_nodes
git clone https://github.com/DanielBartolic/ComfyUI-Qwen3.5.git
pip install -r ComfyUI-Qwen3.5/requirements.txt -r ComfyUI-Qwen3.5/requirements-gguf.txt

ComfyUI Manager installs the pack fine (search "Qwen3.5"), and the Python side is light - the GGUF requirements file is basically comments; you mostly need huggingface-hub, numpy, Pillow, torch. The heavy lift is the binary:

git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j$(nproc)
cp llama.cpp/build/bin/llama-mtmd-cli /usr/local/bin/

Manager won't do this part for you. That's the trap, and it's a one-time cost.

Where people get burned

• llama-mtmd-cli not found - build it (above) or set the cli_path input to your binary. The node checks the input first, then PATH, then /usr/local/bin and a couple of container paths.
• CUDA build sees no devices - common in containers. Pass the arch explicitly: -DCMAKE_CUDA_ARCHITECTURES=120 for Blackwell, 89 for Ada, 90 for Hopper.
• CPU-only - build without -DGGML_CUDA=ON and set n_gpu_layers to 0.
• Models not downloading - drop the .gguf + mmproj-BF16.gguf into ComfyUI/models/LLM/<name>-GGUF/ yourself.