LlamaCPP Unload Model

You've got a local LLM writing your prompt and a big diffusion model about to want every byte of VRAM you own. ComfyUI's automatic memory management will sometimes negotiate that for you, and "sometimes" is why people end up hand-unloading models between steps. LlamaCPP Unload Model does it for you: one node, runs before everything else, waits until the LLM is genuinely gone.

Part of comfyui-llamacpp-plus, which talks to an external llama-server over HTTP rather than loading GGUFs itself.

What it actually does

Two things, and the second is the clever one.

First, the obvious: it POSTs to your llama.cpp server to drop the model. llama.cpp builds don't agree on that endpoint, so the node tries three shapes in order - /models/unload bare, /models/unload with a model field, then /api/v1/models/unload with an identifier - and takes the first one that answers 200 or 204. Total budget for all three attempts is one second, so an unreachable server can't stall your run.

Second: the node has no outputs. No outputs means you can't wire it into the middle of a graph, so you'd normally have no way to say "do this first." ComfyUI runs its output nodes in whatever order a set happens to iterate - arbitrary. So the pack patches the queue. On import, __init__.py wraps PromptQueue.put and moves any LlamaCPPUnloadModel node to the front of that run's execution list. It's not elegant; it is deterministic, and determinism is the whole feature.

Then it waits. The unload acknowledgement only means the teardown is queued, so the node polls /v1/models every half second until the model either disappears from the list or reports a status of unloaded - capped at 30 seconds. Only then does it hand control back, which is what guarantees your 20 GB diffusion loader doesn't start while the LLM is still sitting on the card.

It's also non-idempotent (fingerprint_inputs returns NaN), so it runs on every single queue - no cached-away no-op.

The net effect mirrors keep_alive=0 on the LlamaCPP Connectivity/Chat nodes, but as an explicit point in the graph instead of a hidden side effect.

The input

One, and there's only one:

• connectivity - straight in from the LlamaCPP Connectivity node ("Wire a LlamaCPP Connectivity node here"). The URL and model name come from that node, so there's nothing to configure here at all.

No outputs, no widgets, nothing to set. Drop it anywhere on the canvas - location is genuinely irrelevant, since it gets pulled to the front regardless.

Install

Manager: search comfyui-llamacpp-plus. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/pointave/comfyui-llamacpp-plus

Restart. Dependencies come from requirements.txt (playwright, openai) and get installed by Manager - neither is needed by this node, both belong to the HTML/audio nodes in the same pack. And the prerequisite that catches everyone: you need llama-server already running somewhere. This node doesn't load or host a model, it just tells a server to let go of one.

Where people get burned

"Unload failed" in the console is the one to watch for. If all three endpoint attempts miss - an older llama.cpp build, a different server implementation, a proxy in front of it - the node prints unload failed and then... proceeds. Your run continues with the LLM still resident, and you find out about it as an OOM during the sampler, three nodes later. The console line is the only signal you get.

A run that seems to hang for exactly 30 seconds at the start is this node polling. The early-exit only triggers when the model vanishes from /v1/models or reports status unloaded; a server build that keeps the entry listed without a usable per-model status leaves the loop running to its deadline before the rest of the graph is released. If your queues start with a mysterious half-minute pause, that's the mechanism to look at.

The run-first trick rides on a ComfyUI internal. The patch wraps PromptQueue.put and swallows its own errors, so if a ComfyUI update changes that signature, nothing crashes - it just quietly stops reordering and you're back to arbitrary execution order. If you upgrade ComfyUI and start intermittently OOMing after a working setup, that's the first thing to test.

Don't put this in a chat→generate→chat loop. It's non-idempotent and runs first on every queue, so an iterative workflow pays a full LLM reload per iteration. For that shape, leave the model resident (keep_alive of -1 on the Connectivity node instead of 0) and let the diffusion model take its turn on the card. The unload node is for the one-shot direction: prompt-enhance, unload, then generate at full VRAM.