NInfer - Top K

Top K is the simpler, older cousin of Top P: chop the candidate list to the K most probable tokens and sample from those, always. Where Top P's cutoff moves with the shape of the distribution, Top K's is fixed - which is exactly why Top P tends to be preferred these days, and why Top K is often the one people leave at a default they've never looked at.

The mildly interesting part is that this node has to exist, because the openai Python SDK validates its request kwargs against a fixed list and raises TypeError for anything it doesn't recognise - and top_k isn't on that list. Rather than let the setting die, the pack splits the merged options in two: parameters the SDK knows about are passed normally, and everything else is forwarded through the SDK's extra_body mechanism, whose contents are merged into the top-level JSON request body. So top_k arrives at NInfer looking like any other sampling parameter, and the only trace of the plumbing is that it took a different road.

Inputs and output

• Top K - integer, 0 to 1000, default 20.
• Options - the standard merge, in and out.

No step size, no decimals, and no tooltip - this is the most bare-bones node in the pack.

What it does to your output

top_k = 20 means the model only ever considers the twenty most likely next tokens. For prompt rewriting and structured output, that's a tight, obedient setting and it's a fine default. Raising it towards 100 or more widens the vocabulary, which helps if the model keeps producing flat, repetitive, samey text - the classic symptom of a distribution that's been cut too fine. Lowering it below about 10 tends to read as blunt and clichéd.

The interaction to keep straight: Top K and Top P both truncate candidate tokens, and the tighter one wins. If Top P's nucleus already contains only three plausible tokens, telling it to consider 20 is meaningless. So if you're tuning, that's the argument for changing one at a time - move this, leave Top P at its 0.95 default, and re-run with Force Regen on so you're actually looking at new output.

0 is accepted by the widget and is the obvious "off" value, but whether the server reads it as "disable top-k" or as a literal zero candidates is the server's business and isn't documented here. Don't reach for it as an experiment unless you already have a working baseline to compare against - and if you do, the Extra Body node is a better place to try exotic values, because it doesn't lie to you about what's documented.

Where people get burned

You changed it and nothing happened. For the third time in this pack: check Force Regen on the Chat node. ComfyUI caches node output aggressively, and with Force Regen off an unchanged graph returns the previous completion without calling the model at all. Sampling settings are exactly the kind of change that gets silently cached away.

You typed it into Extra Body instead. That works - same destination, since it's the same extra_body route - but you've now hidden a sampling parameter in a JSON blob, and a later node in the options chain that sets the same key will overwrite it. Use the node.