vLLM Prompt

The name "vLLM Prompt" reads scarier than it is. It doesn't call any API, needs no key, and sends nothing off your machine. You run a small LLM server locally with vLLM, the node sends it a short instruction, and the LLM turns your one-sentence idea into a proper comma-separated Stable Diffusion prompt. Out comes a ready-to-encode combined_prompt string you wire straight into CLIPTextEncode - LLM-assisted prompting without leaving ComfyUI.

It's a niche thing, and honest about being one: if you're happy hand-writing prompts, you don't need it. Reach for it when you want prompt variety at scale - batch runs with wildcards and let the LLM do the rephrasing - or when the prompt blank is staring back at you and you want a first draft instead of a dictionary definition.

How it actually works

The mechanism is refreshingly small. The whole node is one file plus a requests dependency; the heavy lifting happens in the vLLM server you start yourself. On each generation the node:

1. Expands any {option1|option2|option3} wildcards in your prompt client-side, picking one randomly per run. The model always sees a fully resolved string, and multiple wildcards resolve independently.
2. Queries GET /v1/models and takes the first model the server reports - that's how it "detects" your model automatically. Switch models and you must restart the vLLM server for it to pick up the change.
3. Sends your resolved prompt to /v1/completions wrapped in a fixed format: ### Stable Diffusion prompt tags (comma separated, no sentences): followed by Input: ... / Output:. The stop sequence cuts generation at the first newline, so the model can't ramble past a single line of tags.

Then it prepends your prefix to whatever came back. Note that the prefix is not sent to the model - it's pure decoration bolted onto the front of the output. The default (masterpiece, best quality, highres) is classic booru quality-tag vocabulary that still earns its keep on SDXL-lineage anime models like Illustrious, even though it's inert on the newer LLM-encoded checkpoints. The node face shows a live preview with prefix, raw generated text, and the combined string separately.

The inputs that matter

You can leave most of these alone. The two you'll actually touch:

• prompt - your idea, in plain words, wildcards welcome. Default is a dragon example that's worth trying once to see the pipeline working.
• prefix - tags glued to the front of the result. Not sent to the model; keep it as a style anchor.

Everything else is server plumbing: host/port (default localhost:8765, matching the vLLM launch command below), max_tokens (128 is fine for a single line of tags), temperature (drop to 0.3–0.5 if the model starts talking back instead of tagging), and retries (3, for empty or failed responses).

The single output, combined_prompt (STRING), feeds the positive input of CLIPTextEncode → KSampler. The node is also flagged as an output node that always executes, so it generates fresh on every queue - no caching surprises when you change a wildcard.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/OATH-Studio/comfy-vLLM

Then restart ComfyUI. (Or search "comfy-vllm" in ComfyUI Manager's node list and install it there.) Its only Python dependency is requests, so it won't drag a torch-version war into your environment - the one thing you must install separately is the vLLM server itself, and a model. The author's tested stack is vLLM 0.4+ with a Qwen2.5 variant:

vllm serve ./models/Qwen2.5-3B \
    --host 0.0.0.0 \
    --port 8765 \
    --served-model-name Qwen2.5-3B

Model size matters more than you'd think. Qwen2.5-0.5B is genuinely unreliable at instruction-following; 1.5B is usable; 3B is the sweet spot and what I'd run; 32B is overkill for this job but flawless if you've got the VRAM.

Where people get burned

The big one is forgetting this isn't self-contained: no vLLM server on the host/port, and you get a connection error instead of a prompt. Start the server first, confirm curl http://localhost:8765/v1/models answers, then run the node.

Second: conversational filler. If the model answers "Here is a description of a dragon..." instead of tags, lower temperature toward 0.3–0.5, drop max_tokens, or step up to a 3B. The stop-at-newline trick handles most of it, but small models still slip through.

And one worth thinking about before you commit: vLLM is a full serving stack, and if it shares the single GPU ComfyUI is using, you're juggling VRAM between a sampler and a server. People routinely find the LLM step slower than expected versus a lightweight LM Studio or llama.cpp setup on the same card. It shines on a second GPU or a machine with headroom - on a single 8 GB card, a 0.5B model is about all that leaves you room for.