Llama.cpp N-gram Speculative Config

Speculative decoding is one of those tricks that sounds like it should cost you something and mostly just costs you a config node. The idea: while the main model generates token by token, a cheap "draft" guesses several tokens ahead; when the guesses are right, the big model verifies them in one batch and you've skipped several decoding steps. The classic version needs a small draft GGUF. Llama.cpp N-gram Speculative Config is the model-free version - it drafts from repeated token patterns already in your current context, no second model file at all.

The speculative_mode combo is the on/off switch: off preserves normal generation, ngram enables drafting from prompt history. The rest of the inputs tune how aggressively it drafts:

• ngram_size (3) - how many verified context tokens form each lookup key.
• num_pred_tokens (10) - maximum candidate tokens proposed per draft call.
• ngram_mode (k or k4v) - k stores historical positions and uses less memory; k4v caches continuations for cheaper lookup but should have a memory cap.
• ngram_min_hits (2) - minimum historical matches before it proposes anything.
• ngram_max_entries_per_key (8) - 0 passes None (no cap); a cap is recommended for k4v.
• ngram_sync_check_tokens (16) - trailing tokens checked when syncing the incremental history index.

Where this pays off and where it doesn't: n-gram drafting shines when your prompt and generated text repeat patterns - templated output, JSON, lists, code. For free-form creative prose it finds few matches and the win shrinks toward zero, so the honest advice is to leave it off until you're generating something repetitive and then A/B it. Because it's model-free, it works with any GGUF and needs no extra download - that's the appeal versus the native DFlash/DSpark path, which wants a matching draft model and experimental backend support.

The output is the shared speculative socket that feeds the compact Llama.cpp Generate (or Sequential Generate - except Sequential rejects the speculative socket because its decoder history can't be guaranteed independent between resets). It's the same socket the Native Speculative Config node plugs into, and the docs are clear you choose one: n-gram and native modes are mutually exclusive.

Install is the pack standard:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-Ollama-ImageList

Restart ComfyUI; JamePeng llama-cpp-python wheel and ComfyUI 0.19.3+ apply.

A fair summary: this is the low-risk speculative option - no draft model, no experimental fork, and the default off means it can't hurt you unless you turn it on. When it works (structured output, batch captioning with a fixed template), you get a measurable tokens-per-second bump for the price of one socket. When it doesn't, you flip back to off and lose nothing. That's a rare cost-benefit in the LLM-adjacent world, and it's exactly the kind of thing the pack author put a preset on.