🌐 LLM Inference (SQVLM)

The name undersells it. "Simple Qwen-VL" is the workhorse node of this whole pack: a fully local, no-API-key, no-cloud vision-language model runner that loads GGUF models - Qwen3-VL, Qwen3.5, Qwen3.6, Gemma4, even a 1.8B translation model - and makes them talk to the rest of your ComfyUI graph. You feed it an image (or three, or a video, or audio), it writes back text, and that text becomes your caption, your tag list, your refined prompt, or your structured JSON for whatever comes next. Since it runs GGUF, an 8B vision model fits comfortably on a 12GB card. That's the entire pitch: cloud captioning, minus the cloud.

The mechanism is what makes it worth the setup. The node doesn't ship a model and never downloads one - you point it at GGUF files you already have (the README explicitly assumes you've got stuff lying around from LM Studio). You configure it by JSON, either through the config_override input or by saving presets into system_prompts_user.json and picking them from the model_preset dropdown. Under the hood it drives llama-cpp-python, which means the author's obsessions matter: inference runs in a separate subprocess by default so that after the node finishes, llama.cpp leaves nothing in memory. The whole pack exists because the author kept finding other nodes that left VRAM garbage behind, right before video generation needed all of it. If you've ever had an LLM node eat your VRAM and tank your next sampler, you get the appeal.

The inputs that matter

• model_preset - your saved model configs from system_prompts_user.json. This is where model_path, mmproj_path, context size, and sampling live. If you have none yet, start with None and use config_override.
• system_preset - 66 built-in system prompts (Tags, Detailed Description, JSON structure, Style Identification…). Or bypass it entirely with system_prompt_override.
• user_prompt - your actual question, e.g. "Describe this image."
• mode - subprocess (safe, fully cleans up, slowest), direct_clean (same cleanup, no subprocess overhead), keep_vram (model stays loaded for batch runs), plus save1–save3 long-term caches you unload manually.
• config_override - the big multiline JSON. This is where you put model_path, mmproj_path, n_ctx, chat_handler, max_tokens, n_gpu_layers. Stack multiple overrides on top of a preset; later values win.
• image / image2 / image3 / audio / video - up to three+ images, batches supported, plus audio and a frame-reduced video input if the model handles them.

Outputs: text (the answer - wire it anywhere), conditioning (only if you ran the embedding mode), and echoes of system_prompt and user_prompt so you can keep them.

Install

Use ComfyUI Manager (search ComfyUI_Simple_Qwen3-VL-gguf) or:

cd ComfyUI/custom_nodes
git clone https://github.com/KLL535/ComfyUI_Simple_Qwen3-VL-gguf

Restart ComfyUI, refresh with F5. That gets the node itself. The real install, and the #1 place people get stuck, is llama-cpp-python: the stock PyPI build doesn't support Qwen3 at all. You need the JamePeng fork, version 0.3.17+ for Qwen3-VL, 0.3.30+ for Qwen3.5, 0.3.35+ for Gemma4. Prebuilt Windows wheels exist at https://github.com/JamePeng/llama-cpp-python/releases, but they lack AVX/VMM acceleration - for real speed, build from source with CUDA (CMAKE_ARGS=-DGGML_CUDA=on ... pip install .). Budget a few minutes to an hour for that build. Then grab a GGUF model and its mmproj file - vision models need the multimodal projector, and forgetting it is the classic silent failure where nothing outputs.

Common issues

The most reported failure on this exact pack: you follow the README and no text ever appears. It's almost always a bad llama-cpp-python install or a missing mmproj - check the console for import errors at startup, then check your model config. Slow but working? Watch for VRAM overflow in Task Manager's CUDA graph; if shared memory fills up you're swapping and it runs 5–7x slower. Drop n_gpu_layers, set offload_kqv false, or for MoE models use n_cpu_moe to push experts to RAM. Truncated answers mean your context math is off: image_max_tokens + input + max_tokens must fit in n_ctx. And if you enable thinking (enable_thinking), those reasoning tokens eat output budget fast - bump max_tokens to 4096+.

It's fiddly on first setup. Once your first preset is saved, though, it's the most reliable VLM node on the shelf.