ComfyUI Extension: Simple LlamaCPP Client

Authored by ai-joe-git

Created

Updated

3 stars

Run ComfyUI workflows without the setup

No installs, no CUDA version roulette, no GPU sitting idle on your bill. Bring a workflow and run it in the browser.

A lightweight custom node for ComfyUI that connects to a local llama.cpp OpenAI-compatible server with support for chat, vision, streaming, JSON mode, and clean answer/thinking separation.

Looking for a different extension?

Custom Nodes (2)

README

ComfyUI-Simple-LlamaCPP-Client

A lightweight custom node for ComfyUI that connects directly to a local llama.cpp OpenAI-compatible server.

It supports:

  • πŸ’¬ Chat completions (/v1/chat/completions)
  • πŸ–Ό Vision models (image + prompt input)
  • ⚑ Streaming mode (SSE token accumulation)
  • 🧠 Clean Answer + Thinking separation
  • πŸ“¦ Optional JSON-only output mode
  • πŸ”‘ Optional API key authentication
  • πŸŽ› Minimal design (server-side parameters stay server-side)

✨ Features

βœ… Works with any OpenAI-compatible llama.cpp server
βœ… Supports system prompt + user prompt
βœ… Optional image input for multimodal models
βœ… Outputs: Answer, Thinking, JSON, Raw, Model Used
βœ… Auto-detects model from /v1/models
βœ… Clean dropdown UI (no ugly free-text params)

πŸ“¦ Installation

cd ComfyUI/custom_nodes
git clone https://github.com/ai-joe-git/ComfyUI-Simple-LlamaCPP-Client.git

Restart ComfyUI.

πŸš€ Server Example

llama-server.exe ^
  -m Ministral-3-8B-Instruct.gguf ^
  --host 127.0.0.1 ^
  --port 8082 ^
  --mmproj mmproj.gguf ^
  -c 8192

🧩 Node Inputs

| Input | Description | |------|------------| | server_url | llama.cpp server URL (default: http://127.0.0.1:8082) | | prompt | User message text | | system_prompt | Optional system instruction | | image | Optional IMAGE input (vision models) | | api_key | Optional Bearer token |

Model Selection

| Input | Description | |------|------------| | model_mode | Dropdown: auto / custom | | model_override | Only used if model_mode = custom |

Stop Control

| Input | Description | |------|------------| | stop_mode | Dropdown: none, preset:common_eot, preset:triple_hash, custom | | stop_custom | Used only if stop_mode = custom |

Text Cleanup

| Input | Description | |------|------------| | text_postprocess | Dropdown: fix_mojibake, none, ascii_quotes, fix_mojibake+ascii_quotes |

(Default fixes HereҀ™s β†’ Here’s)

πŸ“€ Node Outputs

| Output | Description | |-------|------------| | answer | Final cleaned answer | | thinking | Reasoning if provided | | json | Parsed JSON output (if enabled) | | raw | Full raw server response | | model_used | Model name used |

πŸ“œ License

MIT License.

Run ComfyUI workflows without the setup

No installs, no CUDA version roulette, no GPU sitting idle on your bill. Bring a workflow and run it in the browser.

Learn more