ThinkingLLM Advanced (GGUF)

The plain GGUF node hides llama.cpp's plumbing behind a friendly catalog. The Advanced GGUF node is that plumbing, with a widget for nearly everything the backend accepts - context size, batch sizes, GPU offload depth, image token budget, flash attention, thread counts. It's the node for the person who has read an llama.cpp error message and recognized it. If you're happy with defaults, the non-Advanced version is the better friend; this one exists because sometimes the default ctx isn't big enough or the GPU offload is leaving performance on the table.

Same model catalog as the base GGUF node (70 Qwen3-VL / Qwen3.5 / Qwen2.5-VL / Gemma 4 quants, auto-downloaded on first use), same RESPONSE / RAW_TRACE outputs, same optional image, video, audio, and audio_file_path inputs. The preset_prompt list includes the LTX and Wan scene/timeline templates if you're generating video prompts.

The dials that matter

• ctx - the context window, 32768 by default, up to 262144. Bigger is not free: too large and KV-cache memory eats the VRAM you freed by quantizing. Raise it only when the model truncates.
• gpu_layers - -1 offloads everything it can; 0 is CPU-only. Start at -1, and check RAW_TRACE for the actual offload count.
• image_max_tokens - the token budget per image/video frame. Lower it if multimodal decode fails or VRAM spikes; 4096 is the default.
• n_batch / n_ubatch - prompt-processing batch and physical batch. Higher batch ingests prompts faster but can fail with image/video input; n_ubatch should stay at or below n_batch.
• flash_attn / offload_kqv - on by default, and the node is honest about them: RAW_TRACE reports when the installed backend silently drops the kwarg, so you're not chasing a speedup that was never applied.
• top_k / temperature / top_p / repetition_penalty - the sampler stack. top_k 0 disables it; 20 is the conservative default.
• device - auto/cpu/mps. auto prefers CUDA when PyTorch sees an NVIDIA GPU.
• stream_tokens_to_terminal - your "is it thinking or is it dead" debugger, same as everywhere in this pack.

Two widgets exist purely for legacy workflow compatibility and are ignored by current logic: legacy_seed_mode and legacy_unload_after_run. Unloading is controlled by keep_model_loaded now. Leave them alone.

Installing

Manager install (search ThinkingLLM), or:

cd ComfyUI/custom_nodes
git clone https://github.com/goodguy1963/ComfyUI-ThinkingLLM.git
cd ComfyUI-ThinkingLLM
pip install -r requirements.txt

The GGUF caveats from the base node all apply: this needs a vision-capable llama-cpp-python build (JamePeng's fork), Linux auto-installs a matching wheel on first use, Windows usually needs a manually-matched win_amd64 wheel. Verify with python tools/check_llama_backend.py.

Troubleshooting

The recurring theme with this node is that llama.cpp kwargs get dropped silently by mismatched backends. The node's answer is RAW_TRACE - it logs whether flash attention and offload_kqv actually took, whether GPU offload happened, and the pool/context state. When generation is slow or vision decode fails, read that trace before tweaking pool_size or ctx_checkpoints; most of the time the fix is a correct backend wheel, not a knob. And on high-core/NUMA servers, auto thread counts can be slower - the tooltip's advice to try 8–16 generation threads is the real-world fix when GPU utilization looks low.