LLM Server Draft (GGUF)

This node does nothing on its own, and that's the point. It's one of three optional side-model nodes in Kinburg-Nodes' llm_server package, and it exists so that the settings for speculative decoding only appear in your graph when you're actually using speculative decoding. You wire it into Local LLM Server's draft input and forget about it.

The bigger thing it plugs into

If you haven't met the pack's server nodes yet: they run llama-server or koboldcpp under ComfyUI, behind a small gateway that stays listening while the model itself gets thrown out of VRAM for an image and pulled back for the next message. That's the VRAM problem everyone hits when they want a chat LLM and a diffusion model to share one card - point SillyTavern straight at llama-server and the moment ComfyUI kills that process, the next message hits a closed port.

Local LLM Server carries the whole configuration and emits a base_url you paste into any OpenAI-compatible client. The Draft node is one of its accessories: speculative decoding, which is free speed when it works and a waste of VRAM when it doesn't.

How speculative decoding works

A small, fast model drafts the next handful of tokens; the big model verifies them in one forward pass. Accepted drafts cost one pass for several tokens. Rejected ones cost you the wasted draft, which is cheap because the drafter is small - but only cheap if it's usually right. On a model whose sibling doesn't exist, or on text with no predictable shape, you can end up slower than plain generation. That's the whole tradeoff, and every widget here is about where to sit on it.

The inputs that matter

spec_type picks the method. draft-simple is a small model of the same family; draft-mtp is a Multi-Token-Prediction head shipped alongside the model (an mtp-*.gguf); draft-eagle3 and draft-dflash are the other two real draft kinds. Then there are the ngram-* methods - ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache - which need no draft model at all because they guess from what the text already contains. That's the one to try first. It's free, it costs no VRAM, and on repetitive writing (a template, a long structured prompt, a system-prompt-shaped task) it's surprisingly good. off disables it.

draft_model is the draft .gguf, picked from ComfyUI/models/llm with subfolders, and draft_model_path is the escape hatch when you'd rather paste a path. The draft must share the main model's vocabulary - a different family gets rejected, not silently loaded.

Then the guessing controls, all of which default to "leave the server's own default alone" at 0: draft_n_max is how many tokens to draft per round (the server's own default is 3 - raising it only pays when the draft is usually right), draft_n_min is the fewest worth drafting, and draft_p_min stops drafting once the drafter's confidence drops below a threshold. And draft_n_gpu_layers (-1 = all) keeps the draft on the GPU, which is rather the point of a draft model being small.

One output: draft, a KINBURG_LLM_SERVER_DRAFT bundle. It only fits the server node's draft slot - the three side-model types are named separately so you can't wire a vision config into the draft input by accident.

Install

ComfyUI Manager (search "Kinburg"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/Kinburg/Kinburg-Nodes
# restart ComfyUI

Manager runs the pack's install.py for you, which installs the CUDA-matched llama-cpp-python wheel (0.3.30, picking cu124 or cu130 to match your torch - a mismatch here is the classic "llama.dll or one of its dependencies" failure). Two things the pack does not ship: the server binary (llama-server / koboldcpp - download or build it yourself and drop it under ComfyUI/models/llm) and any model. Draft models are .gguf files like any other; put them in ComfyUI/models/llm.

Gotchas

spec_type, draft_n_min and draft_p_min are llama-server-only. koboldcpp has no equivalent - it infers the method from the draft model you hand it. Rather than dropping your settings silently, the server node lists what it ignored in its status output, so read that before assuming a flag took effect.

A wrong-family draft is a rejection, not a slowdown. Same vocabulary or nothing.

Don't touch the ngram methods' draft_model. Leave it on the placeholder (use model_path field) and don't fill in a path; those methods need no model at all.

Zero means "ask the server", not "zero". Setting draft_n_max to 0 is the default and is not the same as drafting nothing - use spec_type = off for that.