vLLM Samplers

vLLM Samplers is the Spoke node in the comfyui-llm-api-client pack that talks to vLLM - the high-throughput LLM serving engine that's now a Linux Foundation project, and the backend most people run on port 8000. Where the pack's TabbyAPI and YALS nodes handle llama.cpp/ExLlamaV2's anti-repetition samplers, this one covers vLLM's different set: beam search, best_of, ignore_eos, skip_special_tokens, and guided JSON output.

Honest framing first: vLLM is overkill if you just want one local model for yourself. Its selling point is throughput - serving a model to many concurrent consumers without falling over. But if you're already running it (shared setups, structured-output pipelines, anyone who wants a single server answering both ComfyUI and their other apps), this node is the clean way to reach its engine features from inside a workflow.

How it works

Same architecture as the other two Spokes: vLLM Samplers makes no request of its own. It builds a parameter dictionary and feeds it to the pack's Universal OpenAI Hub via the extra_parameters socket, and the Hub sends it along as extra_body on the chat-completions call. That's the OpenAI SDK's mechanism for non-standard, backend-specific parameters.

Wiring: vLLM Samplers → vllm_parameters output → Hub's extra_parameters input. Forget the connection and nothing happens.

The inputs that matter

• use_beam_search + best_of - the pairing that trips people up. Beam search explores several partial sequences and keeps the best-scoring one; it needs best_of greater than 1 to mean anything, and it's markedly slower than greedy/sampling generation. Leave use_beam_search off unless you specifically want it.
• guided_json - vLLM's structured-output knob. Paste a JSON Schema (or a JSON-schema-ish grammar) and vLLM's guided decoding constrains the model to conform. Unlike the YALS node, this one passes the string through as-is - no parsing, no validation on the ComfyUI side, so check your syntax before you send it.
• ignore_eos - keep generating past the end-of-sequence token until you hit max_tokens. Handy for probing, usually wrong for real output.
• skip_special_tokens - defaults to true, which is what you want: it strips the model's control tokens from the reply so you don't get raw <|endoftext|>-style junk in your text.

Note how this node differs from its packmates: top_k defaults to -1, not 0, because -1 is vLLM's native "unset" convention, and there's no XTC/DRY/Mirostat here - those are llama.cpp/ExLlamaV2 ecosystem samplers that vLLM doesn't expose the same way. The required inputs you'll actually touch day to day are guided_json (optional) and the two beam-search toggles; min_p and repetition_penalty are the usual familiar ones.

Output: vllm_parameters (LLM_PARAMS), which connects only to the Hub's extra_parameters.

Installing it

Same single install as the whole pack - no separate download for this node.

cd ComfyUI/custom_nodes/
git clone https://github.com/DBMePls/comfyui-llm-api-client
pip install openai

Restart ComfyUI. The only dependency is openai; torch/numpy/PIL already ship with ComfyUI, and nothing is downloaded because the model lives in your vLLM server. Windows portable: .\python_embeded\python.exe -m pip install openai. ComfyUI Manager: search comfyui-llm-api-client. As with the pack's README, the documented clone URL is a placeholder - use the real one above.

Common issues

• Beam search does nothing. best_of is still 1. Beam search needs best_of > 1; with 1 you're just doing normal (slower) sampling.
• Guided JSON fails on the server. vLLM's structured output requires it to be built with guided-decoding support (the outlines/xgrammar integration), and the schema must be valid. Errors here surface as API Error: in the Hub's response output, and since this node does no client-side validation, the schema is the first thing to double-check.
• Empty model dropdown. The Hub scans ports at ComfyUI startup; if vLLM (8000) wasn't running yet, restart ComfyUI with it up or type the model name into the widget.

If you want vLLM's throughput and the fancy llama.cpp samplers, that's not this node - the pack's TabbyAPI and YALS Samplers nodes handle those, on their own backends.