ialhabbal VLLM GGUF

The name tells you the whole pitch: no transformers, no snapshot_download, no multi-gigabyte sync from HuggingFace on first run. ialhabbal_VLLM_GGUF runs Qwen3-VL and Gemma from local GGUF files through llama.cpp, which is the community's favorite way to keep a vision-language model honest on a consumer card - and the model list leans heavily on the abliterated variants people actually use for uncensored captioning and prompt work.

The dropdown is where the quant lives. You're not choosing "model + quantization" like the HF nodes; the quantization is baked into the filename: Gemma-4-E2B-Uncensored-...-Q8_K_P, Huihui-Qwen3-VL-2B-Instruct-abliterated-Q8_0, Qwen3-VL-8B-Instruct-abliterated-v2.0.Q5_K_M, a Qwen3VL-4B-Instruct-Q4_K_M for the VRAM-strapped, and a couple of plain Qwen3.5 text models. Nine options, all local-only.

How it works

llama.cpp's Llama class loads the model, and a separate mmproj file - the vision projector - lets it actually see images and video. The node detects whether the model is a Gemma or a Qwen and picks the matching chat handler (Gemma4VLChatHandler, Qwen3VLChatHandler, and friends) so the multimodal conversation formats correctly. Video gets frame-sampled the same way as the HF nodes, 16 frames by default.

Inputs are the familiar set: model_name, preset_prompt (same nine built-in instructions), custom_prompt, max_tokens, keep_model_loaded, seed, plus optional image and video. One RESPONSE string out.

The gotcha that bites everyone: you download the files yourself

Read this twice. The GGUF nodes do not download anything. The catalog entries are marked load_local_only, and the download path in the code just raises "download disabled" when a file's missing. You place the .gguf and its matching mmproj-*.gguf into:

ComfyUI/models/llm/GGUF/

The node expects llm/GGUF/<author>/<model>/<file> by default, but it'll also accept the file flat in the folder, or anywhere under models/ - it does a recursive search by filename and takes the first hit. The classic failure is dropping the model file in but forgetting the mmproj; both have to be there or you get a "not found locally" error.

Installation

The pack install is the standard one (ComfyUI Manager → search ComfyUI-ialhabbal, or git clone https://github.com/ialhabbal/ComfyUI-ialhabbal.git into custom_nodes, then restart). The GGUF-specific dependency is llama.cpp itself, and the README is explicit about the recipe:

pip install --upgrade pip setuptools wheel
pip install "llama-cpp-python[server]"

On Windows, a failed install almost always means your Python environment is missing build tools - update pip first and retry. Prebuilt wheels exist for most platforms, so this usually just works.

Common issues

• "GGUF model not found locally" - you skipped the manual download, or the file is named differently than the dropdown entry. Check models/llm/GGUF/.
• "mmproj not found" - the vision projector is a separate file. Forgetting it is the number one reason a "multimodal" node sees nothing.
• Loads but returns garbage - mismatched model + mmproj. A Qwen3-VL projector on a Qwen2.5-VL model won't parse the vision tokens right. Keep them from the same source.
• Everything's slow - Q8 is heavier than Q4; Qwen3-VL-8B-...Q5_K_M is the sensible middle. And if the model doesn't need to see images, the non-multimodal Qwen3.5 entries skip the projector entirely.

It's the node for people who already hoard GGUF files or want the abliterated path. If you'd rather ComfyUI fetch weights for you, the HF-based ialhabbal_VLLM does that.