comfyui-llm-api-client

ComfyUI Universal LLM (Hub & Spoke)

A highly flexible, multimodal, and reasoning-aware custom node suite for connecting ComfyUI to any OpenAI-compatible API.

Instead of cluttering a single node with dozens of backend-specific settings, this suite uses a Hub and Spoke architecture. The "Hub" handles standard text/image generation, while backend-specific "Spoke" nodes inject advanced samplers (like XTC, DRY, and JSON formatting) specifically for TabbyAPI, YALS (llama.cpp), and vLLM.

✨ Features

• 🧠 Native Reasoning Extraction: Automatically parses <think> (DeepSeek-R1) and <|channel>thought (Gemma) tags, outputting the reasoning process and the final response into two separate text strings.
• 👁️ Multimodal Support: Easily pass up to 4 image tensors directly from ComfyUI. The node automatically handles Base64 conversion for Vision-Language Models (VLMs).
• 🔌 Dynamic Model Detection: On ComfyUI startup, the node automatically pings common local ports (5000, 8000, 11434, 1234) and populates the model dropdown for you.
• 🎛️ Advanced Sampler Injection: Use backend-specific Spoke nodes to pass advanced samplers like DRY, XTC, Mirostat, and Guided JSON directly to your LLM engine.

---

🛠️ Installation

1. Navigate to your ComfyUI custom nodes directory:

cd ComfyUI/custom_nodes/

2. Clone this repository (or copy the folder here):

git clone https://github.com/YOUR_USERNAME/ComfyUI-OpenAI-API-Compatible-LLM.git

3. Install the required dependency: This node relies on the official openai Python package. Run the following in your ComfyUI Python environment:

pip install openai

---

🧩 Node Overview

1. Universal OpenAI Hub (Main Node)

Category: Universal LLM This is the core node that communicates with your LLM.

• Inputs: Prompt, System Prompt, API Key, Base URL, standard OpenAI samplers (Temperature, Top P, Max Tokens, Penalties, Seed).
• Optional Inputs: image_1 through image_4 (for vision models), and extra_parameters (to connect a Spoke node).
• Outputs: reasoning_text (if using DeepSeek-R1/Gemma) and response_text.

2. TabbyAPI Samplers (Spoke Node)

Category: Universal LLM/Samplers Designed specifically for the ExLlamaV2 engine via TabbyAPI.

• Features: min_p, top_k, XTC (Probability & Threshold), DRY (Multiplier, Base, Allowed Length), Mirostat, and Dynamic Temperature (max_temp, min_temp).

3. YALS Samplers (Spoke Node)

Category: Universal LLM/Samplers Designed for llama.cpp backends via YALS.

• Features: min_p, top_k, XTC, DRY, Mirostat, and a text input for direct JSON Schema constraints.

4. vLLM Samplers (Spoke Node)

Category: Universal LLM/Samplers Designed for vLLM enterprise setups.

• Features: Beam Search, best_of, ignore_eos, skip_special_tokens, and Guided JSON output generation.

---

🚀 How to Use

Basic Text Generation (Ollama / LM Studio / Local)

1. Add the Universal OpenAI Hub node.
2. Set the base_url to your local provider (e.g., http://127.0.0.1:11434/v1 for Ollama).
3. Type your prompt and generate.

Advanced Generation (TabbyAPI + DRY Sampler)

1. Add the Universal OpenAI Hub node.
2. Add the TabbyAPI Samplers node.
3. Configure the DRY settings on the TabbyAPI node (e.g., multiplier 0.8).
4. Connect the tabby_parameters output to the extra_parameters input on the Hub node.

Vision Processing

1. Load an Image using ComfyUI's standard Load Image node.
2. Connect the IMAGE output to image_1 on the Hub node.
3. Use a prompt like: "Describe this image in extreme detail."
4. Ensure the model you selected in the dropdown supports vision (e.g., LLaVA, Qwen-VL).

---

⚠️ Troubleshooting

Console error: [Universal LLM] ERROR: 'openai' Python package is not installed. You are missing the required python library. If you are using a portable ComfyUI installation, you must install it inside the embedded environment.

• Windows Portable: .\python_embeded\python.exe -m pip install openai
• Linux/Mac: pip install openai

The model dropdown only shows "default_model" and "gpt-3.5-turbo". The node couldn't detect any local LLMs running on ports 1234, 5000, 8000, or 11434 during startup. Ensure your LLM server (Ollama, LM Studio, TabbyAPI, etc.) is running before you start ComfyUI. You can still manually type your model name into the widget by converting it to a text input.