Llama.cpp N-gram Speculative Preset

If you're using the legacy detailed Llama.cpp Generate node, this is your speculative-decoding on-ramp: Llama.cpp N-gram Speculative Preset configures model-free n-gram drafting for that node. It's the same idea as the compact N-gram Speculative Config - the model predicts several tokens ahead using repeated patterns already in the context, and the main model verifies the batch - but it plugs into the legacy node's dedicated ngram speculative socket rather than the compact shared one.

The six inputs are identical in spirit to the compact version, with the tooltips doing heavy lifting:

• speculative_mode - off (default, preserves normal generation) or ngram.
• ngram_size (3) - verified context tokens per lookup key.
• num_pred_tokens (10) - max candidate tokens per draft call.
• ngram_mode - k (stores positions, less memory) or k4v (caches continuations, use a cap).
• ngram_min_hits (2) - minimum historical matches before proposing.
• ngram_max_entries_per_key (8) - 0 = no explicit cap; cap recommended for k4v.
• ngram_sync_check_tokens (16) - trailing tokens checked on history-index sync.

The one sentence to remember from the node description: this does not use a draft GGUF, and it does not touch the Experimental native DFlash/DSpark API. It's backed by llama.cpp's LlamaNGramMapDecoding and it's deliberately kept separate from the native speculative path - the docs explicitly reject mixing the two modes. That separation is the pack's way of guaranteeing that an n-gram preset can never silently start requiring a draft model you don't have.

Where it helps: structured, repetitive generation - JSON, fixed templates, batch captions with a consistent format. Where it's noise: open-ended prose, where patterns repeat rarely and the draft fails so often the verification overhead eats the gains. off is the default for a reason.

One behavioral difference from the compact path worth noting: the legacy Generate node runs n-gram speculative per request, and unlike Sequential Generate there's no cross-item independence guarantee to worry about - so the preset is fine here. The compact path's Sequential node rejects speculative entirely.

Install is the pack standard:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-Ollama-ImageList

Restart ComfyUI; JamePeng llama-cpp-python wheel and ComfyUI 0.19.3+ apply, as with all llama.cpp nodes here.

The pragmatic read: if you're on the legacy node and your workload is templated, wire this preset in and measure. If you're on the compact flow, use the N-gram Speculative Config instead - same trick, shared socket. And if you're tempted by the fancier native speculative path, that's a different node, a different fork wheel, and a different budget of patience; this preset is the version that just works.