Nodes/Multimodal — LLM (Local-first: GGUF + Transformers)/Multimodal — LLM (local first: GGUF + optional transformers)
ComfyUI Node

Multimodal — LLM (local first: GGUF + optional transformers)

**GGUF:** set **GGUF main + mmproj** (.gguf files under models/llm). **Use vision** ON + IMAGE → vision run; **OFF** → text-only (linked image is ignored — Comfy cannot hide the socket, but the model never sees pixels). **Image-only:** vision ON + IMAGE + empty text fields — GGUF sends image only; HF VLM may use a single `.` token only so the HF API accepts the call (not a written instruction). **combined_model** = transformers only (HF id / snapshot folder). Leave **empty** for GGUF-only (including text-only on GGUF). **load_source** applies only to **combined_model**.

By theonetwoone·Created 4 months ago·Updated 4 months ago· 1
Multimodal — LLM (local first: GGUF + optional transformers)
  • image
  • image2
  • image3
  • text
use_visiontrue
load_sourceauto
gguf_main_models_llm— use path fields below —
gguf_mmproj_models_llm— use path fields below —
gguf_model_path
gguf_mmproj_path
gguf_vlm_handlerqwen3-vl
gguf_allow_qwen25_if_qwen3_handler_missingtrue
gguf_n_ctx12288
gguf_streamingfalse
gguf_chat_template_enable_thinkingfalse
include_thinking_in_outputfalse
thinking_open_tag<think>
thinking_close_tag</think>
combined_model
system_prompt
extra_context
prompt
max_new_tokens5000
do_samplefalse
generation_seed_modefixed
generation_seed0
temperature0.70
top_p0.95
top_k20
repetition_penalty1.00
use_repetition_penaltytrue
deviceauto
dtypeauto
trust_remote_codefalse
keep_models_loadedfalse
offload_image_modefalse
log_load_detailstrue
Categoryllm/multimodal

Inputs (36)

NameTypeDefaultDescription
use_visionBOOLEANtrue
load_sourceCOMBOauto3 options: auto, local, huggingface
gguf_main_models_llmCOMBO— use path fields below —1 options: — use path fields below —
gguf_mmproj_models_llmCOMBO— use path fields below —1 options: — use path fields below —
gguf_model_pathSTRING
gguf_mmproj_pathSTRING
gguf_vlm_handlerCOMBOqwen3-vl5 options: qwen3-vl, qwen2.5-vl, gemma3, llava-1.5, llava-1.6
gguf_allow_qwen25_if_qwen3_handler_missingBOOLEANtrue
gguf_n_ctxINT12288512–131072
gguf_streamingBOOLEANfalse
gguf_chat_template_enable_thinkingBOOLEANfalse
include_thinking_in_outputBOOLEANfalse
thinking_open_tagSTRING<think>
thinking_close_tagSTRING</think>
combined_modelSTRING
system_promptSTRING
extra_contextSTRING
promptSTRING
max_new_tokensINT50000–65536
do_sampleBOOLEANfalse
generation_seed_modeCOMBOfixed2 options: fixed, increment_each_run
generation_seedINT00–4294967295
temperatureFLOAT0.700–2
top_pFLOAT0.950–128
top_kINT20-1–100
repetition_penaltyFLOAT1.000.5–2
use_repetition_penaltyBOOLEANtrue
deviceCOMBOauto3 options: auto, cuda, cpu
dtypeCOMBOauto6 options: auto, bfloat16, float16, float32, False, True
trust_remote_codeBOOLEANfalse
keep_models_loadedBOOLEANfalse
offload_image_modeBOOLEANfalse
log_load_detailsBOOLEANtrue
imageoptIMAGE
image2optIMAGE
image3optIMAGE

Outputs (1)

NameTypeDescription
textSTRING