vLLM-T2T丨API

VLLMT2TAPI is the pack's text-to-text LLM node: it sends a chat request to a running server and returns the model's text, the raw response, a status, and a debug dump. The name says "vLLM," but what it actually speaks is the OpenAI chat-completions dialect - so any OpenAI-compatible endpoint, local or hosted, will work.

Display name: "vLLM-T2T丨API".

The mental model: this node has no model

Important clarity, because it's easy to miss: this node runs no model and holds no weights. It's an HTTP client with a JSON payload. The heavy lifting happens on a server you point it at - by default http://127.0.0.1:8091, which is where a local vLLM or vLLM-Omni server would listen. From the graph it looks like a generator; underneath it's POST base_url/v1/chat/completions (external-api-nodes.md is the full map of this pattern). The pattern's tradeoffs apply unchanged: your prompt leaves your ComfyUI process, and per-call latency depends on the server, not your node graph.

How it works

At execution, the node:

1. Builds a messages array - an optional system_prompt message, then the prompt as the user message. If you connect the optional image input, the user message becomes a multimodal content list: the text plus a base64 data-URL of the image, which is how a vision model on the server sees it.
2. Packs the sampling params - temperature, top_p, repetition_penalty, seed, max_tokens - into the request.
3. Merges any advanced_options JSON (the output of VLLMAdvancedOptions) on top.
4. POSTs to <base_url>/v1/chat/completions with your key, and parses the assistant text out of the response.

The key resolution order: the per-node api_key field if filled, else the VLLM_OMNI_API_KEY environment variable, else the QING vllm_omni_api_key setting from ComfyUI's Settings → QING → API config. For a local server that doesn't require auth, leaving it empty is fine (the platform config explicitly allows model fetching without a key).

One implementation detail worth knowing: max_tokens is capped at 2048 in the request, whatever you type. The comment in the source explains why - vLLM-side limits usually govern anyway, and an oversized value gets you a 400. So asking for 128,000 tokens here just silently sends 2048. If you genuinely need more, that's a server-side max_model_len conversation, not a node setting.

Inputs and outputs

• system_prompt, prompt (required) - the conversation.
• platform - "vLLM" only, kept for the pack's shared model-fetch plumbing.
• model - the model id, matching what GET /v1/models returns. Default "default" works if your server registered under that name - set it to the real id.
• base_url - server root, default http://127.0.0.1:8091.
• temperature, top_p, repetition_penalty, seed, max_tokens, timeout_sec - sampling knobs.
• image (optional) - turns the call multimodal.
• api_key (optional), advanced_options (optional).
• Outputs: text (the reply), raw_response (the full HTTP body, sanitized), status ("ok"/"error"), debug_info (a JSON blob with the URL, submitted params, and HTTP status - your first stop when it fails).

Installing it

Ships in ComfyUI-QING:

cd ComfyUI/custom_nodes
git clone https://github.com/GAO-SHIQING/ComfyUI-QING
cd ComfyUI-QING
python install_dependencies.py

Restart ComfyUI, or install "ComfyUI-QING" via ComfyUI Manager. The openai dependency in requirements is present, but this node's own HTTP calls use the standard library.

Troubleshooting

• Status error, debug shows a connection error. Nothing is listening on base_url. Start your vLLM server, check the port, or point base_url at the right host.
• Status error but HTTP status is 404/400. Your model id doesn't match what the server serves, or the endpoint path is wrong. raw_response contains the server's own error message - read it.
• No auth on a local server and it still asks for a key. Some vLLM launches still expect a token. Either pass one via api_key or the env var, or launch the server with auth disabled.
• Output looks truncated. Check max_tokens - remember the 2048 cap in the node, and that server-side limits may be lower.

For the "run a local LLM and have it think for your workflow" job - prompt rewriting, decisions, caption drafting - this is the QING pack's cleanest entry point. The whole local-vs-API decision framework in llm-in-comfyui.md applies: for local uncensored work, a small model behind this node beats a paid API call every time.