llama.cpp Connection

Most of the pack is built around servers it starts and owns. Connection is the escape hatch: it lets you point the whole suite at a llama-server you didn't start - a remote box on your LAN, a dedicated GPU machine, or a llama-server you launched by hand. The pack treats an endpoint it didn't start as externally owned: it will happily generate against it, but it will never implicitly stop it, unload its models, or touch it during Comfy's Unload Models. That's a deliberate safety boundary, and it's the right one.

The second job of this node is tidier workflows: it bundles URL, default model, API-key environment name, TLS policy, and a request deadline into one reusable connection output, instead of re-typing those settings on every prompt node.

How it works

Connection builds a small profile object (LLAMACPP_CONNECTION) from its inputs and hands it to any prompt, Model Info, or Token Count node that accepts a connection socket. When a prompt node uses the profile, it overrides the per-node settings with the profile's values.

The interesting bit is the API key handling. Secrets are never stored in the workflow. The profile stores the name of an environment variable (api_key_env, default LLAMACPP_API_KEY) and the client reads the actual key from the environment at request time. So you can share a workflow JSON without leaking a key, which is a genuine nicety for a pack that otherwise talks to localhost all day. For a remote server with an API key, set the env var before launching ComfyUI:

export LLAMACPP_API_KEY='your-local-key'

The server side of that is the api_key_file option on the start nodes (a file of keys, one per line) - the Connection profile's env var must contain a matching client key.

Inputs that matter

• server_url - the address. Empty means "the managed server," which defeats the point of the node; put a real http://host:port here. For attached endpoints, everything is treated as external.
• model - optional default router model ID, for when you attach to a router.
• api_key_env - which environment variable holds the key.
• verify_tls - off for self-signed certs on a remote box; leave on for anything with a proper cert.
• request_timeout - overall request deadline in seconds.

Output: the connection profile.

When to use it

Three cases justify it: a remote/second-machine llama-server (the "I run a GPU box on the LAN" setup), a hand-started server you don't want the pack managing, and any workflow where the same endpoint settings repeat across many nodes. One caveat that's worth knowing before you commit: an attached endpoint is never implicitly stopped, so VRAM on that remote box is yours to manage - the lifecycle table in the README is explicit that Release and Stop nodes do nothing against an externally owned endpoint. If you want the pack's full start/stop/unload lifecycle, let it own the server instead.