TabbyAPI Samplers

If you serve local LLMs with TabbyAPI, you already know the feeling: the model runs great, the output reads like a broken record, and the fix lives in samplers that a plain OpenAI client will never send. TabbyAPI Samplers is the node that gets those samplers - XTC, DRY, Mirostat, dynamic temperature - from your ComfyUI graph into your TabbyAPI request. It's the thing the pack's main Hub node is waiting for.

What this node actually does

First, the important part: it makes no API call of its own. TabbyAPI Samplers is a "Spoke" node - it builds a dictionary of sampler parameters and hands it to the Universal OpenAI Hub through the extra_parameters socket. The Hub then smuggles those params into the request as extra_body, which is the OpenAI SDK's standard way of passing server-specific knobs through a compatible API.

So the wiring is: TabbyAPI Samplers → tabby_parameters output → Hub's extra_parameters input. Miss that connection and the node does nothing, silently.

TabbyAPI is the local server for ExLlamaV2, the engine that runs EXL2/GPTQ-quantized LLMs fast on consumer GPUs - it's a fixture in the local-LLM scene, the sort of thing a launcher like llm-launcher starts for you. This node assumes you already run it, on port 5000 by default, matching the Hub's startup scan.

The samplers, and the ones you'll actually set

The node's inputs are the modern anti-repetition toolkit, and here's the key gotcha baked into the design: every sampler with a "default" value is skipped unless you change it. The node only emits parameters you've actually set. DRY sits at dry_multiplier: 0 until you raise it - zero means "don't send DRY at all."

• dry_multiplier - the star. DRY (Do-Repetition-Yo) is the penalty that stops a model from repeating itself mid-generation, the R1-era fix for exactly the "broken record" problem. Values around 0.5–0.8 with the default dry_base of 1.75 are a sane starting point. Zero = off.
• xtc_probability + xtc_threshold - XTC (eXclusive Threshold Cutoff) truncates low-probability tokens so the model can't slouch into bland picks. Set xtc_probability above 0 to activate; the threshold defaults to 0.1.
• mirostat_mode - adaptive temperature that targets a perplexity instead of a fixed number. 0 off, 1 or 2 for the two modes, worth experimenting with on longer generations.
• max_temp / min_temp - dynamic temperature range. Leave both at 1.0 for "off"; widen the spread and the temperature wanders between the two.

top_k, min_p and repetition_penalty are the familiar classics. All of it - every input, every default - is exactly what the info_schema says, so what you see in the node is what gets sent.

The single output is tabby_parameters (LLM_PARAMS type). It only wires into the Hub's extra_parameters.

Installing it

The whole pack installs the same way; there's no separate download for this node.

cd ComfyUI/custom_nodes/
git clone https://github.com/DBMePls/comfyui-llm-api-client
pip install openai

Restart ComfyUI. The only dependency is openai (torch/numpy/PIL come with ComfyUI), and no model files are downloaded - the model is whatever TabbyAPI is already serving. ComfyUI Manager works too: search for comfyui-llm-api-client. The README's own clone command is a leftover placeholder URL, so use the real one above.

Common issues

• Nothing changes when you tweak the node. You either didn't connect tabby_parameters to the Hub's extra_parameters, or the parameter you touched was still at its "off" default. Remember: dry_multiplier 0 and xtc_probability 0 mean those samplers are simply not sent.
• The sampler still does nothing. DRY, XTC and friends have to be supported and enabled by your TabbyAPI/ExLlamaV2 build. If you're on an old TabbyAPI, check that the sampler exists server-side before blaming the node.
• Model dropdown shows only default_model. Your TabbyAPI server wasn't running when ComfyUI started, so the Hub's port probe missed it. Restart ComfyUI with the server up, or type the model name into the widget.

The sibling YALS Samplers node in this pack is a near-twin - same samplers, but for the llama.cpp server instead of ExLlamaV2. If your stack is GGUF, that's the one you want.