ComfyUI Node
Llama.cpp Generate (Multimodal)
Loads one local GGUF model, analyzes optional image, audio, and video inputs in one llama-cpp-python chat request, then closes and releases the model immediately. No model cache is retained.
Llama.cpp Generate (Multimodal)
- sampling
- runtime
- ngram_speculative
- images
- audio
- video
- response
- thinking
- raw JSON
- metrics
- media diagnostics
◄model_path[no GGUF models found]►
◄mmproj_path[none]►
◄handlerauto►
◄thinkingfalse►
◄system►
◄prompt►
◄n_ctx8192►
◄max_tokens512►
◄gpu_layersall►
◄temperature0.20►
◄top_p0.95►
◄top_k40►
◄min_p0.05►
◄repeat_penalty1.00►
◄seed-1►
◄stop►
◄n_batch512►
◄override_n_ubatchfalse►
◄n_ubatch512►
◄override_image_max_tokensfalse►
◄image_max_tokens1120►
◄main_gpu0►
◄n_threads0►
◄flash_attentionauto►
◄use_mmaptrue►
◄verbosefalse►
◄reasoning_strengthauto►
◄reasoning_budget0►
◄override_image_min_tokensfalse►
◄image_min_tokens1024►
CategoryOllama/llama_cpp/legacy
Inputs (36)
| Name | Type | Default | Description |
|---|---|---|---|
| model_path | COMBO | [no GGUF models found] | Main model GGUF from ComfyUI's registered LLM paths. The list is not filtered by filename; choose a compatible main model. |
| mmproj_path | COMBO | [none] | Optional multimodal projector GGUF from the same unfiltered list. Choose [none] for text-only requests. |
| handler | COMBO | auto | auto uses the model metadata and the fork's generic MTMD handler. Select a model-specific handler when its template requires one. |
| thinking | BOOLEAN | false | Explicitly request thinking through the selected handler's template control. Gemma 4 uses enable_thinking and Qwen 3 VL uses force_reasoning. A checkpoint that does not support switching may ignore this value. |
| system | STRING | Optional system message sent without trimming or rewriting. | |
| prompt | STRING | User message sent without trimming or rewriting. | |
| n_ctx | INT | 8192512–1048576 | Context window in tokens, including media tokens and output. |
| max_tokens | INT | 5121–131072 | Maximum number of generated tokens. |
| gpu_layers | COMBO | all | Offload all layers, let llama.cpp decide, or use CPU only. |
| temperature | FLOAT | 0.200–5 | — |
| top_p | FLOAT | 0.950–1 | — |
| top_k | INT | 400–10000 | — |
| min_p | FLOAT | 0.050–1 | — |
| repeat_penalty | FLOAT | 1.000–5 | — |
| seed | INT | -1-1–4294967295 | -1 uses llama.cpp's random seed behavior. |
| stop | STRING | Optional single stop string. | |
| n_batch | INT | 5121–65536 | Logical prompt batch size. When an image token limit is overridden, this must be at least the effective limit. |
| override_n_ubatch | BOOLEAN | false | Pass n_ubatch explicitly. Leave disabled to use llama-cpp-python's default. |
| n_ubatch | INT | 5121–65536 | Physical batch size used only when override_n_ubatch is enabled. Gemma 4 vision requires it to cover the selected image token chunk. |
| override_image_max_tokens | BOOLEAN | false | Pass an explicit dynamic-resolution image token ceiling to the MTMD handler. Leave disabled to read the projector's default. |
| image_max_tokens | INT | 11201–65536 | Per-image or per-video-frame token ceiling used only when its override is enabled. n_batch and effective n_ubatch must be at least this value. |
| main_gpu | INT | 00–31 | — |
| n_threads | INT | 00–1024 | 0 lets llama-cpp-python choose the CPU thread count. |
| flash_attention | COMBO | auto | 3 options: auto, enabled, disabled |
| use_mmap | BOOLEAN | true | Memory-map the GGUF while loaded; the mapping is closed after the response. |
| verbose | BOOLEAN | false | Print llama.cpp model, timing, and multimodal chat-handler diagnostics to the ComfyUI console. |
| reasoning_strength | COMBO | auto | Optional reasoning effort hint for templates such as Muse-Glimmer. auto omits the hint so the model template uses its own default. Ignored when thinking is disabled. |
| reasoning_budget | INT | 00–65536 | Maximum reasoning tokens for supported Qwen/Gemma reasoning formats. 0 applies no budget. Ignored when thinking is disabled. |
| override_image_min_tokens | BOOLEAN | false | Pass an explicit dynamic-resolution image token floor to the MTMD handler. Qwen-VL grounding tasks may require 1024 or more. |
| image_min_tokens | INT | 10241–65536 | Per-image or per-video-frame token floor used only when its override is enabled. n_batch and effective n_ubatch must cover this value. |
| samplingopt | OLLAMA_IMAGE_LIST_LLAMA_CPP_SAMPLING | Optional output from Llama.cpp Sampling Preset. When connected, it overrides temperature, top_p, top_k, min_p, and repeat_penalty. | |
| runtimeopt | OLLAMA_IMAGE_LIST_LLAMA_CPP_GEMMA4_RUNTIME | Optional output from Llama.cpp Gemma 4 Runtime Preset. When connected, it overrides the Advanced n_batch, n_ubatch, image_max_tokens, and both override switches. Connect the preset's separate n_ctx and max_tokens outputs to apply those visible values. | |
| ngram_speculativeopt | OLLAMA_IMAGE_LIST_LLAMA_CPP_NGRAM_SPECULATIVE | Optional output from Llama.cpp N-gram Speculative Preset. This model-free mode uses repeated context patterns and no draft GGUF. It is separate from Experimental native DFlash/DSpark decoding. | |
| imagesopt | IMAGE | IMAGE single, batch, list, nested list, or ComfyUI data list. | |
| audioopt | AUDIO | Optional ComfyUI AUDIO single, batch, list, or nested list. Audio is encoded as lossless PCM16 WAV and requires an audio-capable mmproj. | |
| videoopt | VIDEO | Optional ComfyUI VIDEO input. The original encoded stream is passed to llama.cpp's native libmtmd video helper and requires a wheel built with MTMD_VIDEO support plus a compatible mmproj. Embedded audio is not ingested; connect AUDIO separately. |
Outputs (5)
| Name | Type | Description |
|---|---|---|
| response | STRING | — |
| thinking | STRING | — |
| raw JSON | STRING | — |
| metrics | STRING | — |
| media diagnostics | OLLAMA_IMAGE_LIST_LLAMA_CPP_MEDIA_DIAGNOSTICS | — |