LlamaCPP Options

LlamaCPP Options is the sampling override bundle for LlamaCPP Chat. It does nothing on its own - its entire job is to collect a bunch of sampler settings and hand them to the chat node as one LLAMACPP_OPTIONS object. Think of it as a config file that lives in your graph.

The design is the interesting part, and it's a pattern worth internalizing. Every sampling parameter is paired with an enable_* boolean: enable_temperature gates temperature, enable_seed gates seed, and so on. Nothing is sent to the server unless its toggle is flipped on. Leave everything off - the default - and the server just uses its own defaults. That's a genuinely nicer approach than a wall of hardcoded sliders, because it means the node is additive: you override only what you care about, and the model's own settings keep working underneath.

What actually matters

The full list is long, but a beginner sets maybe four or five:

• enable_temperature / temperature - creativity dial. 0.8 is the default; crank toward 1+ for variety, drop toward 0 for robotic consistency. The range goes negative to +10 because llama.cpp lets you, but sane values live between 0 and 2.
• enable_seed / seed - reproducibility. The seed is randomized each time the node is created, which is a small quality-of-life touch. Flip it on with a fixed value and the same prompt gives you the same completion.
• enable_top_p / top_p and enable_top_k / top_k - the two classic nucleus/filtering knobs. Defaults are 0.9 and 40. Most people tune top_p and leave top_k alone.
• enable_repeat_penalty / repeat_penalty - 1.1 default discourages the model from looping the same phrase. Worth knowing because the node maps it to llama.cpp's frequency_penalty under the hood.
• enable_thinking_budget / thinking_budget - for reasoning models (the tooltip names Gemma4 and QwQ). Caps how many tokens the model spends thinking; set the budget to 0 and thinking is disabled entirely. Hand in hand with the think toggle on the chat node.
• enable_num_ctx / num_ctx - context window, mapped to n_ctx. Only touch this if you know your model's max context; asking for more than the server can give just wastes memory.

There's also a video and audio trio that has nothing to do with sampling: video_frame_step, video_max_frames, and audio_max_seconds. They only matter when the chat node runs in video or audio media mode - frame sampling rate, the 60-frame cap, and the 30-second audio trim. And debug prints the whole options dict (and the API payload) to the console, which is the first thing to flip on when a request misbehaves.

How it works

On run, the node filters itself down to only the enabled settings and forwards them to the server. One honest gotcha: the schema accepts enable_min_p / min_p, but if you look at the mapping code, min_p isn't in the list of fields actually forwarded to the server. It's filtered in and then silently dropped, so don't rely on it yet - a plausible future fix, a real gap today.

The single output, options, wires into the options input of LlamaCPP Chat. Skip it and the chat node still works fine; add it and you get control. That's the whole trade, and it's a fair one.