ComfyUI Node
Start llama.cpp Server
Launches and owns a local single-model llama-server process with capability-checked modern options.
Start llama.cpp Server
- server_url
- success
◄modelNo models found - add .gguf files to models/LLM/gguf/►
◄context_size4096►
◄gpu_layers►
◄main_gpu0►
◄port8080►
◄threads►
◄batch_size512►
◄flash_attentionfalse►
◄timeout60►
◄binary_path►
◄host127.0.0.1►
◄tensor_split►
◄no_mmapfalse►
◄flash_attention_modelegacy►
◄mmproj(auto)►
◄sleep_idle_seconds0►
◄api_key_file►
◄api_key_envLLAMACPP_API_KEY►
◄media_path►
◄fit_modeupstream default►
◄unload_comfy_models_before_startfalse►
◄extra_args►
CategoryLlamaCpp
Inputs (22)
| Name | Type | Default | Description |
|---|---|---|---|
| model | COMBO | No models found - add .gguf files to models/LLM/gguf/ | GGUF model file to serve from a configured Comfy model folder. |
| context_size | INT | 40960–1048576 | Maximum context size in tokens. 0 lets llama-server choose. |
| gpu_layers | STRING | Layers to offload to GPU. Empty preserves legacy all-layers behavior; auto, all, or a number are also accepted. | |
| main_gpu | INT | 00–31 | Primary GPU index used by llama-server. |
| portopt | INT | 80801–65535 | TCP port for the owned llama-server. |
| threadsopt | STRING | CPU generation threads. Empty lets llama-server choose. | |
| batch_sizeopt | INT | 5121–65536 | Logical prompt-processing batch size. |
| flash_attentionopt | BOOLEAN | false | Legacy flash-attention toggle. Prefer flash_attention_mode for new workflows. |
| timeoutopt | STRING | 60 | Startup readiness deadline in seconds. Empty has no limit. |
| binary_pathopt | STRING | Explicit llama-server executable. Empty uses LLAMA_SERVER_BINARY, then PATH. | |
| hostopt | STRING | 127.0.0.1 | Network interface address to bind. Loopback is safest. |
| tensor_splitopt | STRING | Per-GPU model proportions, for example 3,1. | |
| no_mmapopt | BOOLEAN | false | Disable memory-mapped model loading. |
| flash_attention_modeopt | COMBO | legacy | Modern flash-attention mode. Legacy preserves the released flash_attention widget behavior. |
| mmprojopt | COMBO | (auto) | Matching multimodal projector for local VLMs. (auto) leaves projector discovery to llama-server. |
| sleep_idle_secondsopt | INT | 00–86400 | 0 disables idle sleep. Positive values release model VRAM while idle. |
| api_key_fileopt | STRING | Path to a llama-server API key file, one key per line. | |
| api_key_envopt | STRING | LLAMACPP_API_KEY | Environment variable containing the matching client key. The secret is not serialized. |
| media_pathopt | STRING | Directory allowed for llama-server file:// media inputs. | |
| fit_modeopt | COMBO | upstream default | Control whether llama-server adjusts unset arguments to fit device memory. |
| unload_comfy_models_before_startopt | BOOLEAN | false | Evict Comfy-managed models before llama-server allocates GPU memory. |
| extra_argsopt | STRING | Advanced llama-server arguments. No shell is used. |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| server_url | STRING | URL of the running owned llama-server. |
| success | BOOLEAN | Whether server startup reached its ready state. |