🚀 Remote VLM Business Task

This is the node the pack's quick-start is built around. Remote VLM Business Task is the one that takes a single image, sends it to whatever vision model you've configured (LM Studio, Ollama, a llama.cpp server, or any OpenAI-compatible endpoint), and returns text - with the task picked from a dropdown so you don't have to write prompts by hand. If you install this pack for exactly one node, this is probably it.

The README's recommended remote flow is dead simple:

Remote Vision Model Config → Remote VLM Business Task ← IMAGE

The config node on the left describes where your model lives and what it's called; this node does the actual request. It's an output node, so its result STRING shows up in the UI and is ready to wire anywhere.

What the dropdown gives you

• Caption - detailed caption, uses the same template as Caption Prompt Builder.
• OCR - text extraction, template from OCR Prompt Builder.
• Prompt Reverse - write a generation prompt for the image, template from Prompt Reverse Builder.
• JSON - structured output with a default schema (subject, action, scene, style, text, tags).
• Custom - ignores the built-ins and uses whatever you type into custom_prompt.

The template choice matters more than it looks: each mode embeds the right instruction so you don't have to remember whether Qwen wants "describe in detail" or "output JSON only, no Markdown."

The inputs that matter

model_config is the required REMOTE_VISION_MODEL connection from the config node - no config, no run. Then the three you'll actually touch:

• task - the five-mode dropdown above.
• max_tokens - default 1024, up to 8192. Bump it for long JSON or verbose captions; keep it low for tags.
• temperature - default 0.2, which is low on purpose. For structured/OCR work you want determinism, not creativity. Raise toward 0.7 only if you're doing freeform captioning and want variety.

image (optional) takes the IMAGE you're analyzing; custom_prompt overrides the whole task template when set. timeout defaults to 300s - vision models are slow, and the tooltip's advice of 300–600s for image workloads is not a joke.

How it works, honestly

The node checks the config's service_available flag (set when the config node pings your server), then builds the image as a base64 data URL and POSTs it to your endpoint over the OpenAI-compatible protocol. No API key is required if you're talking to a local server - the "remote" in the name is a lie, it means "not a local GGUF load", and it can point at 127.0.0.1. If the service isn't up, you get a friendly "❌ 服务不可用" string back instead of a crash, which is genuinely nicer than most ComfyUI nodes.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt

Restart ComfyUI, find it under GGUF-VLM → 🚀 Business. This node needs the requests dependency (in requirements.txt) plus a running server on the other end - the pack itself doesn't ship a model.

Troubleshooting

The classic failure is "service unavailable" - check that your LM Studio/Ollama server is actually running on the port in the config node (1234 LM Studio, 11434 Ollama). A second one: the dropdown model list comes from the config node's Refresh button, so a model you just downloaded may not appear until you hit refresh. And if you get text back that ignores the task - you asked for JSON and got prose - the model or its system prompt is overriding the instruction, so check the system prompt on the config node and consider that a low quant flattens instructions.