Local LLM (GGUF)

A ComfyUI node in Kinburg-Nodes/LLM with 7 inputs and 10 outputs.

By Kinburg·Created 2 months ago·Updated 3 days ago· 1

Local LLM (GGUF)

◄user_prompt►

◄system_override—►

◄grammar_override—►

◄unload_after_runconfig default►

◄live_previewfalse►

CategoryKinburg-Nodes/LLM

Inputs (7)

Name	Type	Default	Description
config	KINBURG_LLM_CONFIG		Wire a 'Local LLM Settings (GGUF)' node here — it carries the model, system prompt, sampling, reasoning, output format, etc.
user_prompt	STRING		The prompt / question for the model.
imageopt	IMAGE		Optional image(s) for vision — needs an mmproj on the Settings node (Vision Settings). Omit for a text-only run.
system_overrideopt	STRING		Optional: replaces the config's system_prompt for this node (connect-only). Context still applies.
grammar_overrideopt	STRING		Optional: a GBNF grammar (connect-only) that replaces the config's grammar and forces gbnf_grammar output for this node.
unload_after_runopt	COMBO	config default	Free the model from VRAM after THIS node runs, without touching the shared config. 'config default' follows the Settings node; 'unload after run' frees VRAM (a different model runs next); 'keep loaded' stays warm (the same model runs next).
live_previewopt	BOOLEAN	false	Stream the generated text to a 'Kinburg Live Log' node as it's written, token by token. Grammar / JSON runs stream too, so a card built via grammar_override types itself out as the model constrains it.