Nodes/LM Studio Image to Text Node for ComfyUI/LM Studio (Structured Output)
ComfyUI Node

LM Studio (Structured Output)

Make an LLM hand you clean JSON — and split it into fields ComfyUI can use

By mattjohnpowell·Created 2 years ago·Updated 3 months ago· 59
LM Studio (Structured Output)
  • image
  • json_output
  • value_1
  • value_2
  • value_3
  • value_4
  • value_5
  • value_6
text_inputDescribe this image.
json_schema{ "type": "object", "properties": { "subject": {"type": "string"}, "style": {"type": "string"}, "mood": {"type": "string"}, "tags": {"type": "array", "items": {"type": "string"}} }, "required": ["subject", "style", "mood", "tags"] }
output_keyssubject style mood tags
system_promptYou are a helpful AI assistant. Always respond with valid JSON.
model_keygemma-3-4b-it-qat
auto_unloadTrue
unload_delay0
seed-1
max_tokens1000
temperature0.70
debugfalse
timeout_seconds300
strip_thinkingtrue

Most LLM nodes in ComfyUI hand you a blob of text and wish you luck. The Structured Output node is the one that doesn't. It forces the model to reply with JSON that matches a schema you define, then pulls the fields you care about out onto separate output pins. Instead of parsing prose with string surgery, you get a prompt enhancer whose result is already split into subject, style, mood, and tags - each on its own wire, ready to plug into different parts of your graph. If you're building anything that feels like a pipeline rather than a single lucky render, this is the pack's most underrated node.

The trick is that the JSON constraint isn't a vibe or a prompt instruction - it's enforced by the LM Studio SDK itself. The node builds a chat with the model (optionally attaching an image), then sends the request with a structured config carrying your parsed schema. The server constrains generation to valid JSON matching that schema, so what comes back is parseable, every time. The node then walks the top-level keys you listed and exposes each as a string output, joining array values with ", " for you.

The inputs that actually matter

  • text_input - the user message. Default is "Describe this image." so it works as an image captioner out of the box.
  • json_schema - the whole point. A JSON Schema object that the model must match. The default (subject, style, mood, and a tags array) is a sensible starting shape for prompt-building; edit it to whatever your workflow needs.
  • output_keys - a newline-separated list of top-level keys to extract, in order, into value_1 through value_6. Only six, only top-level, and array values get comma-joined. If your schema has more fields than that, the raw JSON on json_output still has everything.
  • system_prompt - defaults to "Always respond with valid JSON," which is a good habit even though the schema is doing the real work.
  • model_key - default gemma-3-4b-it-qat. It's a text model; if you're passing an image, switch to a vision one.
  • seed, temperature, max_tokens, strip_thinking, auto_unload/unload_delay - same dials as the rest of the pack. strip_thinking matters more here: a thinking model's <think> blocks are not valid JSON and the regex cleanup handles them, but a low temperature plus a small model is the more reliable path to valid output.

Outputs

Seven pins: json_output (the full JSON string, for your eyes or for a parser) and value_1value_6 (the extracted field strings). Wire value_1 into a positive prompt, value_2 into a negative prompt, drop value_3 into a filename suffix - that's the workflow this node exists for.

One honest warning: if your JSON schema is malformed, the node throws before the request even goes out. Keep the schema simple (flat objects, string and string-array values) and it behaves; try to get clever with nested objects and you'll spend your evening in the console. A schema error tells you exactly which field broke, at least.

Install and troubleshooting

Standard pack install: ComfyUI Manager (search "LM Studio"), or git clone https://github.com/mattjohnpowell/comfyui-lmstudio-image-to-text-node into custom_nodes plus pip install lmstudio, then restart ComfyUI. LM Studio must be running with the Server started and a model loaded - the lmstudio SDK connects to localhost:1234 automatically, so no host/port config unless you're feeding the legacy ip_address/port inputs from an old workflow.

If the model returns garbage instead of JSON: turn temperature down, check that the model you picked supports the SDK's structured output, and enable debug to see the exact request in the ComfyUI console. And if you're coming off an LM Studio upgrade and hitting bosToken errors, that's an SDK/app version mismatch: pip install lmstudio --upgrade (or run upgrade_lmstudio.py in the pack folder) and restart ComfyUI completely.

CategoryComfyExpo/LMStudio

Inputs (14)

NameTypeDefaultDescription
text_inputSTRINGDescribe this image.The user message / prompt sent to the model.
json_schemaSTRING{ "type": "object", "properties": { "subject": {"type": "string"}, "style": {"type": "string"}, "mood": {"type": "string"}, "tags": {"type": "array", "items": {"type": "string"}} }, "required": ["subject", "style", "mood", "tags"] }A valid JSON Schema object. The model will be constrained to return JSON matching this schema.
output_keysSTRINGsubject style mood tagsNewline-separated list of top-level JSON keys to extract into value_1 … value_6 outputs (in order). Array values are joined with ', '.
system_promptSTRINGYou are a helpful AI assistant. Always respond with valid JSON.
model_keySTRINGgemma-3-4b-it-qat
auto_unloadCOMBOTrue2 options: True, False
unload_delayINT00–3600
seedINT-1-1–18446744073709550000
imageoptIMAGE
max_tokensoptINT10001–4096
temperatureoptFLOAT0.700–2
debugoptBOOLEANfalse
timeout_secondsoptINT30010–3600
strip_thinkingoptBOOLEANtrueStrip <think>...</think> reasoning blocks from the response (for models with thinking mode enabled).

Outputs (7)

NameTypeDescription
json_outputSTRING
value_1STRING
value_2STRING
value_3STRING
value_4STRING
value_5STRING
value_6STRING