vLLM Generator

VLLMGenerator is where the pilcothink pack steps up from "small model on your box" to "whatever big model your vLLM server is serving." You feed it a VLLM_CONNECTION from the vLLM connective node, a system prompt, a user prompt, and optionally an image, and it calls the server's OpenAI-compatible chat endpoint and returns the answer as a STRING. If you've got a vLLM setup for other work, this is the cleanest way to reuse that horsepower inside a ComfyUI graph instead of running a second model stack.

The pack's bundled vision models top out around 4B parameters. A vLLM server, by contrast, can be hosting a 70B-class model with proper batching - so the practical difference is quality. The trade-off is that you have to stand up and manage vLLM yourself; this node is just the client.

How it works

It POSTs to /v1/chat/completions with an OpenAI-style messages array. A system prompt becomes a system message; your prompt and, if present, the image go into a multimodal user message - the image is converted from the ComfyUI tensor to a base64 data URL, which is the standard way to send vision to OpenAI-compatible servers. On the way back, it separates reasoning from the answer the same way the Ollama generator does: it checks for a reasoning_content or reasoning field on the message first (that's the vLLM-native channel for thinking tokens), and falls back to parsing <think>...</think> out of the content. You get clean text and thinking outputs.

The inputs that matter

• connection - from "vLLM Connective." Required, force-input.
• system_prompt - optional system message, multiline, default empty.
• prompt - the user message; default "Describe the image." since this node is vision-aware.
• temperature (0.7), top_p (1.0) - sampling knobs.
• max_tokens - 512 default, up to 8192.
• image (optional) - an IMAGE tensor; first frame only, sent as a PNG data URL.

Outputs: text (the answer) and thinking (the reasoning) - both STRING.

Install

Part of the comfyui_pilcothink_VisionSLM pack:

cd ComfyUI/custom_nodes
git clone https://github.com/gpdev-Pilcothink/comfyui_pilcothink_VisionSLM

Restart, or install via ComfyUI Manager (search comfyui_pilcothink_VisionSLM). The generator needs only requests at runtime; the vLLM server itself is a separate, sizable setup.

Gotchas

The connection node's model selection is validated against what's loaded on the server - if the model isn't loaded, the generative call will 404 or error. So vllm serve <model> before you try to generate. If your server requires an API key, it was configured on the connective; forgetting it shows up as a 401 on the generate call. And the thinking split depends on how the served model reports reasoning: models that stream reasoning_content are clean; models that only produce <think> blocks get the regex treatment, and badly formatted output can leak reasoning into text. Check your first output before you trust it downstream.

Same ecosystem caveat as the rest of this pack: it's brand new with no track record, so install from the official repo. If you only want the vLLM path and don't care about the bundled model nodes, the heavy dependencies still come along for the ride - that's just how this pack ships.