ThinkingLLM (GGUF)

The plain ThinkingLLM node runs Qwen-VL through Transformers in fp16, which costs a lot of VRAM. This one runs the same models through llama.cpp as GGUF quants, which is how you fit a vision LLM next to a diffusion model on a mid-range card. The default Qwen3-VL 4B Q4_K_M is about 2.4 GB instead of 7.5 GB, and on a card that's already hosting a checkpoint and a VAE, that difference is the whole ballgame.

If you've been around the GGUF world since the Flux era, none of this is exotic: quantized weights, a Q ladder, Q8 being basically fp16 at half the size. The pack just hands you a curated catalog (gguf_models.json) of Qwen3-VL, Qwen3.5, Qwen2.5-VL, and Gemma 4 GGUFs with pre-matched mmproj vision files, auto-downloads what's missing on first use, and auto-discovers local .gguf files you drop in models/LLM/GGUF/ - as long as the matching mmproj sits beside them.

How it works and what to set

Everything important mirrors the HF node: model_name (with [installed] / [local] markers in the list), preset_prompt / custom_prompt, max_tokens, keep_model_loaded, seed, stream_tokens_to_terminal, enable_thinking, auto_finalization_retry, hf_token, plus optional image, video, and audio inputs. Outputs are RESPONSE and RAW_TRACE.

The two you'll actually touch on day one:

• model_name - pick the quant that fits. Q4_K_M is the accepted compromise; Q8 is nearly lossless if you have the room. Bigger quant = slower prompt processing but better answers, and llama.cpp quants beat Transformers fp16 on memory every time.
• enable_thinking - on by default. On Qwen GGUFs it works; on non-Qwen GGUF models the tooltip is blunt about it: advisory, may not be honored by the backend.

Installing - read this twice

Manager install (search ThinkingLLM) gets you the node, but the GGUF path needs a vision-capable llama-cpp-python, and the stock PyPI package usually doesn't ship the Qwen3VLChatHandler / Qwen25VLChatHandler the vision models need. On Linux, the node auto-checks at first GGUF use and tries to install a matching JamePeng backend wheel; on Windows you'll likely need to grab a matching win_amd64 wheel from JamePeng's releases yourself (match your ComfyUI Python version - cp310/cp311/cp312/cp313 - and build type), per docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md in the repo.

cd ComfyUI/custom_nodes
git clone https://github.com/goodguy1963/ComfyUI-ThinkingLLM.git
cd ComfyUI-ThinkingLLM
pip install -r requirements.txt

You can probe your backend before blaming anything else:

python tools/check_llama_backend.py

Common issues

Where people get burned: installing the GGUF node, running it, and seeing vision silently fail because llama-cpp-python is the CPU-only PyPI build. If RAW_TRACE says GPU offload is no/unknown, verify the wheel before hunting in your workflow. This node also exposes optional audio inputs even though normal Qwen models can't hear - the read-only info box will warn you about that; audio really belongs to the dedicated Gemma 4 Audio node. And since this is a GGUF loader, treat the download step as part of the run: first use can stall while it fetches the model and mmproj.