NInfer - Top P

Top P is nucleus sampling: instead of considering every token in the vocabulary, the model sorts them by probability and keeps the smallest set whose probabilities add up to top_p, then samples from that set. Drop it to 0.5 and it's only ever choosing between the tokens that make up the top half of the distribution.

Set against Temperature, the difference is granularity. Temperature flattens or sharpens the whole distribution - it changes how likely unlikely words are. Top P cuts a tail off - it removes the possibility entirely. Which is why the two are usually mentioned together and why tuning both at once is a mistake: you lose track of which one changed the output.

Inputs and output

• Top P - float, 0.0 to 1.0, step 0.01, default 0.95.
• Options - the standard merge, in and out.

One mechanical note that's worth more than it looks: top_p is one of the parameters the openai Python SDK accepts natively, so the Chat node passes it through as an ordinary request argument. top_k is not, which is why that one travels a different route through extra_body. You don't have to care - the node handles it - but it explains why top_k shows up in the pack's source while top_p doesn't.

Should you touch it?

Mostly no. 0.95 is a sane default and it's what the server expects people to use. The tail-cutting argument for lowering it - "I don't want the model picking weird tokens" - is real but overlaps with what Temperature already does, and Temperature is the knob with more predictable behaviour and more accumulated advice behind it.

Where Top P earns its place: structured output. If the model has to emit JSON, a specific tag format, or a fixed template, narrowing the nucleus is a decent way to make malformed output less likely without going all the way to greedy decoding. Something like 0.8 with a low temperature is a reasonable pairing for that job. The community's stronger answer for the same problem is to ask the server for structured output directly (a response_format key through the Extra Body node), which constrains the shape rather than just making it more probable.

The other case is the opposite: when you're generating variants and want them genuinely different from each other, raising Top P slightly while leaving Temperature alone opens the vocabulary back up. As always, pair it with the Seed node if you want to be able to come back to a result you liked.

Two things to avoid. Don't set it to 0.0 expecting a cleaner answer - that's degenerate and the server's behaviour there is its own business. And don't tune it against a graph where the Chat node's Force Regen is off; you'll change this widget, re-run, get the identical cached reply, and conclude the node does nothing. It doesn't do nothing. It just didn't run.