Ino Vllm Run Sync

Every serious local setup ends up wanting an LLM in the loop at some point - captioning, prompt rewriting, error diagnosis. Running one locally is a GPU tax on top of your diffusion workload, which is why the cloud route exists. Ino Vllm Run Sync calls a Runpod serverless vLLM endpoint from inside a workflow and waits for the answer, optionally handing the model an image URL so you can do vision tasks without a local vision model. The description: "Runs a synchronous vLLM inference request on Runpod serverless with optional image input."

This is a niche node in a niche pack - it's the whole Runpod category - but for people who already rent Runpod GPUs, it's the cleanest way to get an LLM answer into the graph.

How it works

It submits a chat-completion-style request to your endpoint, then polls until the run finishes - that's what "sync" means here: the node blocks (within its timeout) rather than returning a job id for you to check later. It retries failed polls up to max_failed_retries times, so a cold-starting serverless worker doesn't instantly kill your workflow. It returns the full result envelope: success flag, job id, status, timings, the response text, any reasoning output, and the finish reason.

The inputs that matter

• url, api_key, model - your endpoint address, key, and model name. Each falls back to an env var if left empty: RUNPOD_LLM_URL, RUNPOD_LLM_API, RUNPOD_LLM_MODEL. Note the README mentions RUNPOD_API_KEY, but the code reads the RUNPOD_LLM_* set - set those if you're going the env-var route.
• user_prompt - the actual request (required).
• system_prompt - optional system instructions.
• image_url - optional; pass a URL and the endpoint gets a vision input.
• temperature (0–2, default 0.7) and max_tokens (default 1024) - the usual sampling knobs.
• timeout (default 600s), max_polls (default 70), poll_delay (default 10s) - how long it waits before giving up.

Outputs: success, id, status, delay_time, execution_time, response, reasoning, finish_reason.

Where it wires in

Caption-and-clean pipelines (image_url out of a download node, response into a prompt builder), LLM-assisted debugging via InoTerminalLog, or any workflow that needs a judgment call made before generation continues. The response string is the one you'll feed onward.

Installing it

Part of ComfyUI-InoNodes. ComfyUI Manager → search "ComfyUI Ino Nodes" → install → restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/nobandegani/ComfyUI-InoNodes
cd comfyui_ino_nodes
pip install -r requirements.txt

Restart after. You need a Runpod serverless endpoint running vLLM and its API key; the openai client library (a pack dependency) handles the request plumbing.

Common issues

The dominant failure is a cold worker. Serverless endpoints spin down when idle, and the first call can take tens of seconds to come up - if success is false with a timeout, raise max_polls or poll_delay rather than assuming the endpoint is dead. Second, the env-var names are easy to get wrong: use RUNPOD_LLM_* (what the code reads), not RUNPOD_API_KEY. And remember this blocks your workflow for the full generation - a long max_tokens on a slow worker can pin the run for minutes. For heavy use, that's a feature of serverless billing (you pay per run), not a bug.