ComfyUI Node

LM Studio (Image to Text)

A local vision model that lives on your GPU

By mattjohnpowell·Created 2 years ago·Updated 3 months ago· 59
LM Studio (Image to Text)
  • image
  • Description
user_promptDescribe this image in detail
system_promptThis is a chat between a user and an assistant. The assistant is an expert in describing images, with detail and accuracy
model_keyqwen/qwen3-vl-8b
auto_unloadTrue
unload_delay0
seed-1
max_tokens1000
temperature0.70
debugfalse
timeout_seconds300
strip_thinkingtrue
model
ip_address
port0

The "Image to Text" node is the reason this pack exists, and it's the one you'll actually reach for. Give it an image - from a Load Image node, a batch of renders, anything that resolves to an IMAGE tensor - and it returns a plain-text description of what a vision model sees. Caption a reference photo so another model can reuse it. Turn a generated image into a prompt for a different checkpoint. Build a "describe → re-prompt → regenerate" loop that runs on your own hardware. No API key, no cloud account, nothing leaves your machine.

That last bit is the whole pitch. The node doesn't run a model itself - LM Studio does. ComfyUI just holds the door open: the node converts your image tensor into a temporary JPEG, hands it to LM Studio's server over the official lmstudio Python SDK (localhost:1234 by default), and waits for the vision model to talk back. It's the same relationship as every other node in this pack: LM Studio must be running with the Server tab started and a vision model loaded, or nothing happens.

The inputs that matter

You only really set a handful:

  • image (required) - the image you want described. Wire it from Load Image or anywhere you've got an IMAGE output.
  • user_prompt - what you ask about the image. Default is "Describe this image in detail", and it's usually right.
  • system_prompt - defaults to "an expert in describing images, with detail and accuracy." This is where you steer output style: ask for booru tags, a single-line prompt, or a raw description.
  • model_key - the LM Studio identifier of a vision model. Default is qwen/qwen3-vl-8b, which is a good default and a genuinely decent captioner, but you have to actually download it in LM Studio first.
  • auto_unload / unload_delay - memory management. Leave auto_unload on; unload_delay of 0 unloads the model right after the answer, a few hundred seconds keeps it warm if you're looping.

Beyond that: max_tokens (default 1000, capped at 4096), temperature (0.7 - drop it toward 0.2 for stable captions), timeout_seconds (300), and strip_thinking, which defaults to on and cuts <think>...</think> reasoning blocks out of the reply. Vision models with thinking mode enabled will happily dump their chain-of-thought into your caption otherwise. seed behaves like everywhere else in ComfyUI: -1 for random, anything else to reproduce.

The single output, Description (STRING), feeds straight into a CLIP Text Encode for a text-to-image pass, into another LLM node for further processing, or into a Text/ShowText node so you can actually read it.

Install and first run

Install via ComfyUI Manager (search "LM Studio"), or the usual:

cd ComfyUI/custom_nodes
git clone https://github.com/mattjohnpowell/comfyui-lmstudio-image-to-text-node
pip install lmstudio   # the pack auto-installs it at load if missing

Then restart ComfyUI. In LM Studio: download a vision model (qwen3-vl-8b is the pack's default), go to the Server tab, load it, and click Start Server. The node connects automatically - there's no IP/port field you need to fill, though older workflows' model, ip_address, and port inputs still exist for backward compatibility (they just print a deprecation warning).

Where people get burned

The top three, in order: the LM Studio server isn't running (it must be - no server, no output, just an error), the model_key you typed doesn't match anything in your LM Studio library, and you pointed it at a text-only model that can't see the image. Enable debug on the node and watch the ComfyUI console - it logs exactly which model it's using and what went wrong. If you see Object missing required field 'bosToken' type errors after an LM Studio upgrade, the SDK and app are out of step: pip install lmstudio --upgrade (or run the pack's upgrade_lmstudio.py), then restart ComfyUI completely.

This is a local-LLM integration, which is a category worth a moment of caution given the ecosystem's history with LLM-vision nodes - install from the official repo or via Manager, and don't grab a fork from a random GitHub link. Sourced correctly, this is one of the handiest captioning tools in ComfyUI.

CategoryComfyExpo/I2T

Inputs (15)

NameTypeDefaultDescription
imageIMAGE
user_promptSTRINGDescribe this image in detail
system_promptSTRINGThis is a chat between a user and an assistant. The assistant is an expert in describing images, with detail and accuracy
model_keySTRINGqwen/qwen3-vl-8b
auto_unloadCOMBOTrue2 options: True, False
unload_delayINT00–3600
seedINT-1-1–18446744073709550000
max_tokensoptINT10001–4096
temperatureoptFLOAT0.700–2
debugoptBOOLEANfalse
timeout_secondsoptINT30010–3600
strip_thinkingoptBOOLEANtrueStrip <think>...</think> reasoning blocks from the response (for models with thinking mode enabled).
modeloptSTRING
ip_addressoptSTRING
portoptINT00–65535

Outputs (1)

NameTypeDescription
DescriptionSTRING