ComfyUI Node
Local LLM (server client, text)
A ComfyUI node in Kinburg-Nodes/LLM with 34 inputs and 11 outputs.
Local LLM (server client, text)
- text
- thoughts
- finish_reason
- sys_tokens
- user_tokens
- output_tokens
- gen_seconds
- help
- thoughts_tokens
- answer_tokens
- server_log
◄backendllama-server (launch)►
◄server_binary▾►
◄server_binary_path►
◄base_url►
◄model▾►
◄model_path►
◄model_name►
◄system_promptYou are a helpful assistant.►
◄user_prompt►
◄context►
◄max_tokens512►
◄temperature0.70►
◄top_p0.95►
◄top_k40►
◄min_p0.00►
◄repeat_penalty1.10►
◄stop►
◄seed0►
◄n_ctx4096►
◄n_gpu_layers-1►
◄host127.0.0.1►
◄port8080►
◄extra_args►
◄strip_thinktrue►
◄answer_marker►
◄thinking_directivemodel default►
◄custom_directive►
◄output_formattext►
◄grammar►
◄ready_path►
◄startup_timeout180►
◄request_timeout600►
◄unload_comfy_modelstrue►
◄keep_alivetrue►
CategoryKinburg-Nodes/LLM
Inputs (34)
| Name | Type | Default | Description |
|---|---|---|---|
| backend | COMBO | llama-server (launch) | llama-server / koboldcpp are launched for you; 'connect to running server' just calls base_url of a server you already run. |
| server_binary | COMBO | Executable for the launch backends. Auto-lists ones in ComfyUI/models/llm; else pick the placeholder and set server_binary_path. Download it yourself — not bundled. | |
| server_binary_path | STRING | Full path to the server executable when the dropdown is the placeholder. | |
| base_url | STRING | For 'connect to running server': the server root, e.g. http://localhost:5001. Blank = use host:port below. Ignored by the launch backends. | |
| model | COMBO | Main .gguf from ComfyUI/models/llm (launch backends). Placeholder = type a path in model_path. | |
| model_path | STRING | Full path to the .gguf when 'model' is the placeholder. | |
| model_name | STRING | Value sent as the request's 'model' field. Usually optional; some servers route by it. Blank = 'local'. | |
| system_prompt | STRING | You are a helpful assistant. | — |
| user_prompt | STRING | — | |
| context | STRING | Reference material appended to the system prompt (e.g. character cards from Context Collector). Empty = ignored. | |
| max_tokens | INT | 51216–32768 | — |
| temperature | FLOAT | 0.700–2 | — |
| top_p | FLOAT | 0.950–1 | — |
| top_k | INT | 400–32768 | — |
| min_p | FLOAT | 0.000–1 | Min-p sampling. 0 = off. Try ~0.05 (often paired with top_p=1.0, top_k=0) |
| repeat_penalty | FLOAT | 1.101–2 | — |
| stop | STRING | Stop strings, one per line. | |
| seed | INT | 0-1–18446744073709550000 | — |
| n_ctx | INT | 4096256–1048576 | Context size for the launch backends (llama-server --ctx-size / koboldcpp --contextsize). Raise it if answers get cut off (finish_reason = length). |
| n_gpu_layers | INT | -1-1–1000 | GPU layers for the launch backends (llama-server -ngl / koboldcpp --gpulayers). -1 = all. |
| host | STRING | 127.0.0.1 | Host for launch backends (also the base_url fallback for connect mode). |
| port | INT | 80801–65535 | Port for launch backends. koboldcpp's usual default is 5001. |
| extra_args | STRING | Raw CLI flags appended to the launch command, e.g. `--spec-type draft-mtp` (llama-server) or `--draftmodel path` (koboldcpp). Changing this relaunches the server. | |
| strip_think | BOOLEAN | true | Keep reasoning out of 'text' (it still goes to 'thoughts'). Off = leave raw reasoning in 'text'. |
| answer_marker | STRING | For models that reason WITHOUT <think> tags: answer = text after the LAST line equal to this marker; before it -> thoughts. Empty = use <think> tags. | |
| thinking_directive | COMBO | model default | Append a reasoning-control directive to the prompt. /no_think makes Qwen3-style models skip the <think> phase; 'custom' uses the field below. |
| custom_directive | STRING | Directive appended when thinking_directive = custom (e.g. /no_think). | |
| output_format | COMBO | text | Output: free text · valid JSON (json_object) · custom GBNF grammar (field below) · ideogram4_json = built-in nested Ideogram JSON. |
| grammar | STRING | GBNF grammar text, used when output_format = gbnf_grammar. Sent to the server as its 'grammar' field. | |
| ready_path | STRING | Override the readiness probe path. Blank = per-backend default (llama-server /health, koboldcpp & connect /v1/models). | |
| startup_timeout | INT | 18010–3600 | How long to wait for the server/model to be ready. |
| request_timeout | INT | 60010–7200 | — |
| unload_comfy_models | BOOLEAN | true | Unload ComfyUI (image) models from VRAM before generating. |
| keep_alive | BOOLEAN | true | Keep a launched server running after generation (holds VRAM). Off = stop it after each run. No effect in connect mode. |
Outputs (11)
| Name | Type | Description |
|---|---|---|
| text | STRING | — |
| thoughts | STRING | — |
| finish_reason | STRING | — |
| sys_tokens | INT | — |
| user_tokens | INT | — |
| output_tokens | INT | — |
| gen_seconds | FLOAT | — |
| help | STRING | — |
| thoughts_tokens | INT | — |
| answer_tokens | INT | — |
| server_log | STRING | — |