LocalLLM Unload (passthrough)

The premise is simple and slightly painful: you can absolutely run a 24B quantized LLM on the same card you generate images with - just not at the same time. Krea 2 at FP8 wants 8–12 GB, SDXL wants its own 10-ish, and a Q4_K_M 24B wants 14.4 GB. Something has to leave before the sampler starts, and this node is the eviction notice.

What it does

Nothing visible. It takes a text input, returns that same text unchanged, and in between it throws away the language model.

[LocalLLM Prompt Generator] --text--> [LocalLLM Unload] --text--> [CLIP Text Encode] --> [KSampler]

The passthrough is the whole design. Because the string travels through the node, ComfyUI can't run the text encode - or anything downstream of it - until this node has finished, so you get a guaranteed "unload happens before sampling" ordering without touching execution settings. Put it off to the side, dangling, and it does nothing useful; it has to sit in the path.

Underneath it's a call that closes every cached Llama instance, drops the reference, runs gc.collect() and torch.cuda.empty_cache(), and logs [LocalLLM] All models unloaded. That's it.

Why you can't just use the usual buttons

The model cache in this pack is its own dictionary, keyed on the model path, the mmproj, n_ctx and n_gpu_layers. It is not registered with ComfyUI's model management, so the things you'd normally reach for - Manager's "free model cache", the Unload Models button - don't know it exists. Until this node (or the unload_after_run flag) runs, that VRAM is parked, and the model load it's hiding behind is tens of seconds for a 24B, not a hiccup.

Using it well

The unload_after_run flag is the simpler answer. Setting it to true on the Prompt Generator or Image Caption node frees the model the instant generation finishes - same effect, no extra node. Reach for the Unload node when you want the unload at a later, explicit point in the graph, typically right before the sampler while the encode is still downstream.

Two traps:

• It unloads everything. One call clears the whole cache, so if your graph has two LLM nodes loaded, both are gone.
• Don't put it in the middle of a chain. Caption → prompt generator is the standard two-LLM workflow, and both nodes share one cached instance for exactly this reason. Drop an Unload between them and you force a full reload on the second call - you've made the graph slower, not leaner.

And if you're iterating on the prompt itself, don't unload at all. Reloading a 24B between every test run is the most common way people conclude that local LLMs are unusably slow.