🌐 Remote Vision Model Config (LM Studio/Ollama)

The local GGUF vision path in this pack is locked to Qwen2.5-VL and needs a working llama-cpp-python build. The remote path throws all of that away: if you've got a vision model running in LM Studio, Ollama, or Nexa, this node just points ComfyUI at it and lets your graph caption and analyze images using whatever the server has loaded - Qwen3-VL, LLaVA, whatever. For Windows users especially, the README's recommendation is unambiguous: LM Studio, because it bundles its own runtime and downloads models through a GUI, so you skip the Python/CUDA setup entirely.

How it works

Same pattern as the remote text config, but aimed at vision. Set api_type (LM Studio, Ollama, Nexa SDK, or OpenAI Compatible), set base_url, and click the 🔄 Refresh Models button to populate model from the server's /v1/models endpoint. The defaults in the tooltips are the ports you want: LM Studio 1234, Ollama 11434, Nexa 8080.

There's an optional system_prompt that defaults to "You are a helpful assistant that describes images accurately and in detail." - you'll probably want to leave or customize that depending on whether you're captioning for fun or building a training dataset. The single output is model_config (REMOTE_VISION_MODEL), which plugs straight into the 🔍 Remote Vision Analysis node.

Where it shines

The whole point is delegation. Your local GPU stays free for diffusion; the vision model runs in a separate server, possibly on a different machine, possibly a beefier one. On a 6 GB card that can't hold a VLM plus a checkpoint, this is the difference between captioning locally and not captioning at all. The setup is genuinely two steps - start the server with a vision model loaded, fill in base_url - which is why this is the path the author pushes for non-technical Windows setups.

Install and the two things that bite

Installation is the pack clone-and-pip, and notably it does not require the CUDA llama-cpp build - the server does the heavy lifting:

cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt

Two traps. First, vision models are slow, and the companion analysis node defaults to a 300-second timeout for exactly that reason - if analysis keeps failing, that's the timeout you need to raise, not a server crash. Second, the model list only shows what the server currently has loaded. On LM Studio, a "downloaded but not loaded" model won't appear until you load it in the app. Empty dropdown almost always means the server isn't running, the port's wrong, or the model isn't actively loaded.