ComfyUI Node
ThinkingLLM Advanced (GGUF)
A ComfyUI node in ThinkingLLM with 33 inputs and 2 outputs.
ThinkingLLM Advanced (GGUF)
- image
- video
- audio
- RESPONSE
- RAW_TRACE
◄model_nameQwen3-VL-4B-Instruct-c_abliterated-v2.Q4_K_M.gguf [~2.4GB]►
◄deviceauto►
◄preset_prompt🖼️ Detailed Description►
◄custom_prompt►
◄max_tokens8192►
◄temperature0.60►
◄top_p0.90►
◄repetition_penalty1.00►
◄frame_count16►
◄ctx32768►
◄n_batch512►
◄gpu_layers-1►
◄image_max_tokens4096►
◄top_k20►
◄pool_size4194304►
◄keep_model_loadedfalse►
◄seed1►
◄legacy_seed_modefixed►
◄legacy_unload_after_runfalse►
◄n_ubatch512►
◄n_threads0►
◄n_threads_batch0►
◄flash_attntrue►
◄offload_kqvtrue►
◄ctx_checkpoints0►
◄stream_tokens_to_terminalfalse►
◄enable_thinkingtrue►
◄auto_finalization_retryfalse►
◄hf_token►
◄audio_file_path►
CategoryThinkingLLM
Inputs (33)
| Name | Type | Default | Description |
|---|---|---|---|
| model_name | COMBO | Qwen3-VL-4B-Instruct-c_abliterated-v2.Q4_K_M.gguf [~2.4GB] | GGUF vision model from gguf_models.json or auto-detected local files. [installed] means the catalog model file was found on disk; [local] means an uncatalogued local model. Missing GGUF or mmproj files are downloaded on first use. |
| device | COMBO | auto | auto prefers CUDA when PyTorch sees an NVIDIA GPU. If RAW_TRACE says GPU offload is no or unknown, verify your llama-cpp-python CUDA wheel before blaming the model. |
| preset_prompt | COMBO | 🖼️ Detailed Description | Select 'No preset' to use only the custom prompt or image input. |
| custom_prompt | STRING | Additional user input that gets combined with the preset template. Leave empty to use only the template. | |
| max_tokens | INT | 819264–32768 | Maximum new tokens to generate. Larger values give more room for reasoning but increase runtime and memory use. |
| temperature | FLOAT | 0.600–2 | Sampling randomness. Lower values are more deterministic; higher values are more varied. |
| top_p | FLOAT | 0.900–1 | Nucleus sampling cutoff. Lower values restrict token choice; 0.9 is a balanced default. |
| repetition_penalty | FLOAT | 1.000.5–2 | Values above 1.0 reduce repeated phrases; 1.0 leaves repetition unmodified. |
| frame_count | INT | 161–64 | Number of video frames to sample. More frames improve video context but raise image-token, batch, and context pressure. |
| ctx | INT | 327681024–262144 | llama.cpp context window. Too large can reduce speed and increase KV-cache memory even on strong GPUs. |
| n_batch | INT | 51264–32768 | Prompt processing batch size. Higher can improve prompt ingestion but may raise memory use or fail with image/video inputs. |
| gpu_layers | INT | -1-1–200 | Number of model layers to offload to GPU. -1 asks llama.cpp to offload all possible layers; 0 is CPU-only. |
| image_max_tokens | INT | 4096256–1024000 | Upper token budget for each image/video frame. Lower it if multimodal decode fails or VRAM use is too high. |
| top_k | INT | 200–32768 | llama.cpp sampler top-k. 0 disables top-k filtering; 20 is a conservative default. |
| pool_size | INT | 41943041048576–10485760 | llama.cpp memory pool size for multimodal work. Increase only when backend errors point at pool/context capacity. |
| keep_model_loaded | BOOLEAN | false | Keep the GGUF model in RAM/VRAM after the run so repeated prompts skip model loading. Disable if you need memory back for other nodes. |
| seed | INT | 11–4294967295 | Sampling seed. The node also uses fixed-seed prompt persistence, so identical inputs can reuse the saved result. |
| legacy_seed_mode | COMBO | fixed | Legacy workflow compatibility only. This widget is ignored by current ThinkingLLM logic. |
| legacy_unload_after_run | BOOLEAN | false | Legacy workflow compatibility only. Model unloading is controlled by keep_model_loaded. |
| n_ubatch | INT | 5120–32768 | Physical batch size. Keep at or below n_batch. Lower values can improve stability; 0 uses min(n_batch, 512). |
| n_threads | INT | 00–256 | CPU generation threads. On high-core/NUMA servers, auto can be slower; try 8-16 if GPU utilization is low. |
| n_threads_batch | INT | 00–256 | CPU prompt/batch threads. Tune separately from generation threads on server CPUs. |
| flash_attn | BOOLEAN | true | Enable llama.cpp flash attention when the installed backend accepts and supports it. RAW_TRACE reports if the kwarg was dropped. |
| offload_kqv | BOOLEAN | true | Keep K/Q/V and KV-cache related work on GPU when supported. RAW_TRACE warns when the backend drops this kwarg. |
| ctx_checkpoints | INT | 00–32 | Checkpoint count for multimodal context handling. JamePeng builds usually recommend 0 for single-turn ComfyUI runs. |
| stream_tokens_to_terminal | BOOLEAN | false | Print generated tokens live in the ComfyUI terminal. Useful for long runs and backend troubleshooting. |
| enable_thinking | BOOLEAN | true | Enable model reasoning/thinking when the backend supports it: True=allow thinking, False=force direct answer. Even when enabled, easy prompts may still get a direct answer, and this node automatically disables thinking when there is not enough output budget left for useful reasoning. For non-Qwen GGUF models this is advisory. |
| auto_finalization_retry | BOOLEAN | false | If enabled, runs an extra LLM completion when the first output is empty or reasoning-only. Disabled by default so one node execution performs one generation pass. |
| hf_token | STRING | Optional Hugging Face access token for private or gated GGUF/mmproj downloads. It is passed only to the download call, never logged or cached, and the in-memory copy is dropped after the download attempt. Clear this field before saving or sharing workflows. | |
| imageopt | IMAGE | — | |
| videoopt | IMAGE | — | |
| audioopt | AUDIO | — | |
| audio_file_pathopt | STRING | Optional local audio file path. M4A, MP3, WAV, FLAC, and other FFmpeg-readable files are decoded to 16 kHz mono WAV before inference. |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| RESPONSE | STRING | — |
| RAW_TRACE | STRING | — |