Hosted VLM API (Secure)

Sometimes your local GPU can't run a model good enough for the job, and that's what this node is for. Hosted VLM API sends your image (or a sampled batch of video frames) to a cloud vision model and returns the text - OpenAI, Gemini, Claude, Grok, DeepSeek, Groq, Mistral, Together, OpenRouter, or a custom OpenAI-compatible endpoint like vLLM or Ollama. The pack's local Modern VLM node covers the small stuff; this covers "I need GPT-5.6-class vision on a deadline" without buying a bigger card.

The "Secure" in the display name isn't marketing. Keys are never node inputs - the workflow only contains a provider selection, and the server resolves that provider's fixed environment variable (OPENAI_API_KEY, GEMINI_API_KEY, ANTHROPIC_API_KEY, XAI_API_KEY, CUSTOM_API_KEY, and so on) at execution time. Before anything is uploaded, your image is resized, JPEG-compressed, and bounded: max_image_edge (default 1536), jpeg_quality (88), and max_frames (8, max 32) control how much actually leaves your machine, and the node enforces per-image and total request limits. Built-in credentials only ever talk to the provider's official HTTPS host. Redirection and env proxies are off by default, and provider exceptions get redacted before they hit your log. This is the safest hosted-node design in the pack and it shows.

The inputs that matter: model_name is the preset picker (19 choices, default OpenAI GPT-5.6 Sol), prompt and system_prompt are what you'd expect, and image_detail (auto/low/high) trades tokens for fidelity. output_format is where it gets interesting - Text, or JSON object / JSON Schema, which is locally parsed and validated so invalid output fails the node instead of poisoning your automation. That's genuinely useful for feeding structured results into the pack's JSON Extract tooling. web_search turns on provider-side server search where supported (off by default, and it can add cost - don't flip it on for content you don't want processed under the provider's search terms). stream_output pipes tokens into a connected View Text node.

Outputs are text (the answer), model_used (which preset resolved), and frames_sent (how many video frames actually went up - handy for API billing math).

Installing it

Same as the whole pack - ComfyUI Manager (search "ComfyUI VLM nodes"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

Then set the env var for whichever provider you're using (e.g. export OPENAI_API_KEY=...) in the environment that starts ComfyUI and restart the server. The requirements bring in openai and httpx, so this node needs no extra manual installs.

Common gotchas

• OpenRouter/Custom need model_override - it's mandatory for those, optional for presets. The dropdown can't know every model ID on OpenRouter, so you type it.
• Custom/local endpoints: remote URLs must be HTTPS; keyless HTTP is restricted to localhost/loopback. If you're pointing at a local Ollama, credential_source = "No key (loopback custom endpoint only)".
• Your key isn't being read? It's an environment problem, not the node. The key lives server-side, so a ~/.bashrc export on your shell won't help if ComfyUI was launched by a service or a different user.
• The node fails loudly on unsupported presets rather than pretending - a model that can't do web search errors before the request instead of silently not searching.

One honest note: hosted VLMs cost money per request, and the node won't hide that from you - frames_sent exists precisely so you can see the bill coming. For high-volume captioning, the local ModernVLM node is the frugal choice; this is the "I need the best answer, not the cheapest" choice.