Local LLM (GGUF)

The standing community want in the LLM-in-ComfyUI space has been "run any GGUF directly as a node, no separate Ollama server in the loop." Local LLM (GGUF) is that: a text (and optionally vision) generation node that loads a .gguf with llama-cpp-python inside ComfyUI, runs your prompt, and hands you clean outputs - text, thoughts, token counts, the lot. The thing that makes it worth using over the dozens of other local-LLM nodes is VRAM discipline: inference runs in a separate worker process, so when it exits, the OS reclaims all of its memory. Perfect right before image generation, which is usually when you want an LLM's help the least.

The config bundle

You don't set model options on this node. Everything - model, system prompt, sampling, context, reasoning, output format - lives on the pack's Local LLM Settings (GGUF) node and arrives through the single config input (a KINBURG_LLM_CONFIG bundle). One Settings node can drive several LLM nodes at once, which is the point. The per-node things you set here:

• user_prompt - the question. The one thing you'll change every run.
• image (optional) - for vision, needs an mmproj on the Settings node. Omit for text-only.
• system_override - connect-only; replaces the config's system prompt for this node only.
• grammar_override - connect-only; a GBNF grammar that forces structured output for this node.
• unload_after_run - config default follows Settings, unload after run frees VRAM after just this node, keep loaded stays warm.
• live_preview - streams tokens to a Kinburg Live Log node as they're written.

The outputs that matter

text is the answer; thoughts is the reasoning, split out separately (with strip_think on the Settings node, reasoning stays out of text entirely - the failure mode of an LLM leaking its scratch-work into your prompt is real, and this is the fix). Then the telemetry: sys_tokens, user_tokens, output_tokens, a thoughts_tokens/answer_tokens split of the output, gen_seconds, and finish_reason. There's also a help output - the pack's own cheat-sheet, wire it to a Preview-as-Text node to read it.

How it earns its place

The local-vs-API decision in this space mostly comes down to "small and obedient beats large and clever" - rewriting a rough idea into a structured prompt doesn't need a frontier model, and an 8B Qwen on your own card is uncensored, offline, and free per call. GGUF is what lets a chat model share a consumer card with a diffusion model at all - Q4_K_M is the accepted compromise for a 12GB card, and this node makes the quantized format a first-class citizen of the graph.

Install

Install the pack via ComfyUI Manager (search "Kinburg-Nodes"), or cd ComfyUI/custom_nodes && git clone https://github.com/Kinburg/Kinburg-Nodes. The LLM nodes need llama-cpp-python (CUDA build); Manager runs the pack's install.py automatically, which picks the wheel matching your torch's CUDA version. By hand with ComfyUI's Python:

<ComfyUI>/.venv/Scripts/python.exe <ComfyUI>/custom_nodes/Kinburg-Nodes/install.py

Drop your .gguf into ComfyUI/models/llm. The classic gotcha: the wheel's CUDA major must match torch's - a cu124 wheel on a CUDA-13 torch fails with cudart64_12.dll not found. The install script exists precisely to stop that from biting you.