ComfyUI LM Studio Bridge

LM Studio Bridge is a single, powerful ComfyUI node that connects to LM Studio’s native REST API. Run LLMs and vision models locally and use them directly in your image‑generation workflows – for prompt enhancement, image captioning, style analysis, or any text‑to‑text / image‑to‑text task.

🚀 Key features – automatic model detection, adaptive image resizing, PNG optimisation, keep‑alive HTTP sessions, and full control over generation parameters.

---

📦 Installation

1. Navigate to your ComfyUI custom_nodes directory

   cd /path/to/ComfyUI/custom_nodes
   

2. Clone this repository

   git clone https://github.com/caradat/comfyui-lmstudio-bridge.git comfyui-lmstudio-bridge
   

3. Install Python dependencies
   Activate your ComfyUI environment (venv/conda) and run:

   pip install requests Pillow numpy
   

   (If a requirements.txt is present, you can use pip install -r requirements.txt instead.)

4. Restart ComfyUI – the node will appear under ComfyExpo/REST as LM Studio Bridge.

---

🚀 Getting Started

1. Prepare LM Studio

• Download and launch LM Studio.
• Load a model (for image‑to‑text, ensure it’s a vision model, e.g. qwen35, gemma4, etc.).
• Go to the Developer tab (<->) and start the server (default port 1234).
• (Optional) Note the model_key shown in LM Studio if you want to specify a model manually.

2. Use the Node in ComfyUI

• Right‑click → Add Node → search for LM Studio Bridge.
• Connect:
  • prompt (required) – your main text instruction.
  • image (optional) – connect an image if you use a vision model.
  • model_key (optional) – leave empty to auto‑detect the loaded model.
• Set generation parameters (temperature, max_output_tokens, etc.).
• Run the workflow – the generated text appears at the Generated Text output.

---

🔧 Node Reference

LM Studio Bridge

| Input | Type | Default | Description | |-------|------|---------|-------------| | prompt | STRING | "Make the ready-to-use prompt with the image description:" | Main text prompt. | | system_prompt | STRING | "You are a helpful AI assistant." | System instruction for the model. | | host | STRING | "localhost" | LM Studio server hostname or IP. | | port | INT | 1234 | LM Studio server port. | | max_output_tokens | INT | 2000 (min 1, max 4096) | Maximum tokens in the response. | | temperature | FLOAT | 0.6 (0.0–2.0, step 0.1) | Randomness – lower = more deterministic. | | image | IMAGE (optional) | – | Image input for vision models. | | model_key | STRING (optional) | "" | Specific model ID. If empty, the first loaded model from LM Studio is used automatically. | | debug | BOOLEAN | False | Enable verbose logging in the ComfyUI console. | | timeout_seconds | INT | 300 (10–3600) | Request timeout. |

Output:

• Generated Text (STRING) – the model’s response.

Note on IS_CHANGED: The node includes a hash of all inputs (including image data when present). This ensures that ComfyUI re‑executes the node whenever any parameter or the uploaded image changes.

---

🧠 Automatic Model Detection

If you leave model_key empty, the node:

1. Queries LM Studio’s /api/v1/models endpoint.
2. Finds the first model that has at least one loaded instance (field loaded_instances > 0).
3. Uses that model ID for the generation.

This makes switching models in LM Studio seamless – no need to update the node. In debug mode, the detected model ID is printed to the console.

---

🖼️ Image Optimisation

To reduce token usage and network latency (vision models encode images into tokens), the node automatically:

• Resizes images if either dimension exceeds 1536 px (aspect ratio preserved).
• Converts RGBA → RGB when the alpha channel is fully opaque.
• Saves as PNG with optimize=True and compress_level=6.

These steps typically reduce image size by 50‑80% without noticeable quality loss.

---

🔁 Persistent HTTP Session

The node uses a single requests.Session() for all calls. This enables HTTP keep‑alive and connection reuse, improving performance when the node is executed multiple times in a workflow.

---

📡 API Format

This node uses LM Studio’s native REST API (not the OpenAI‑compatible endpoint).

• Endpoint: /api/v1/chat
• Payload format (text-only):

  {
    "model": "...",
    "input": "your prompt",
    "system_prompt": "...",
    "max_output_tokens": 2000,
    "temperature": 0.6
  }
  

• For vision models, input becomes an array of objects:

  "input": [
    {"type": "text", "content": "Describe this image"},
    {"type": "image", "data_url": "data:image/png;base64,..."}
  ]
  

Make sure your LM Studio server is using the native API (the default when you start the server).

---

🛠️ Troubleshooting

| Problem | Likely Fix | |---------|-------------| | Error: Cannot connect to LM Studio server at host:port | LM Studio must be running and the server started (button in the Server tab). | | Error: The LM Studio server is accessible, but no models are known. Load a model in LM Studio and try again. | Load at least one model in LM Studio before starting the server. | | Error when requesting the model list: ... | Check network connectivity or firewall. The node uses HTTP GET to /api/v1/models. | | Image encoding failed | The input image tensor may be invalid. Try using a Load Image node and connect it directly. | | Error: Request to LM Studio server failed: ... | Inspect the full error in the console; enable debug=True for more details. | | Response is empty or No content generated. | The model may not have produced any output. Check your prompt and system prompt. | | Timeout after X seconds | Increase timeout_seconds or check system load (some large models can be slow). |

Enable debug = True to see detailed logs:

• Model detection process
• Payload (truncated for large images)
• HTTP response status
• Preview of the generated text

---

📚 Example Workflow

1. Load an image (e.g., a photograph of a landscape).
2. LM Studio Bridge node:
   • prompt: "Describe this image in one short sentence for use as a Stable Diffusion prompt."
   • Connect the image to the image input.
   • model_key: leave empty (auto‑detects your loaded vision model).
   • temperature: 0.4 (more focused).
3. Connect the Generated Text output to a Show Text node.

Run the workflow – the model writes a caption. Then you can copy that caption into a positive prompt for Stable Diffusion.

For a full visual example, see workflow.png in the repository.

---

📄 License

This project is licensed under the MIT License – see the LICENSE file for details.

---

🙏 Acknowledgements

• ComfyUI – the incredible node‑based UI.
• LM Studio – making local LLMs easy to run and serve.
• Built by caradat.