LLM (Managed Cache, Advanced)

If you've ever tried to put a local LLM in front of your diffusion pipeline, you know the dance: the LLM eats your VRAM, and then your Flux or SDXL sampler can't fit. LLMOptionalMemoryFreeAdvanced is the pack's full-control answer - a text-only chat node with every llama.cpp knob exposed, plus an unload switch that frees the model after it answers so your actual generation step gets the GPU back.

What it is

This is the advanced LLM sampler in gokayfem's VLM Nodes pack, in the same family as LLMSampler but self-contained: it loads the GGUF directly from a dropdown instead of taking a model from a separate loader node. Text in, text out. The "Memory Optional" bit is the point - you decide whether the model stays resident (fast repeat calls) or gets released after each run (free VRAM for the rest of the graph).

The inputs that matter

• ckpt_name - dropdown of GGUF files in models/LLavacheckpoints. Any GGUF LLM works - a Qwen prompt-gen model, an IF prompt model, whatever you've downloaded.
• system_msg - default "You are a helpful AI assistant." This shapes everything; write a real system prompt and you'll get dramatically better outputs than the default.
• prompt - your input. Multiline.
• gpu_layers - GPU offload count (default 27). Raise it if the model crawls on CPU; the VRAM it uses is exactly what unload gives back later.
• max_ctx - context length, default 4096.
• n_threads - CPU threads for non-GPU work.
• temperature - default 0.1. The creativity dial; for prompt generation you'll usually want it low.
• top_p / top_k / frequency_penalty / presence_penalty / repeat_penalty / seed - the full sampling panel. Defaults are fine to start; repeat_penalty (1.1) is the one to raise when small models start looping.
• unload - the star. Check it to release the model from memory after generation.

Output is a single STRING.

How it works

Straightforward llama.cpp inference: the node loads the GGUF, runs your system message plus prompt through it with the sampling parameters applied, and returns the text. With unload off, the model stays cached for the next call - that's the right mode for iterative prompt tweaking. With it on, the model frees after each run, which is the right mode for a one-shot "generate a prompt, then sample" workflow on a tight GPU.

Install

The usual pack install:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes

Restart ComfyUI (or Manager → "VLM Nodes"). Python 3.9+ and llama-cpp-python required. Text-only GGUFs go in models/LLavacheckpoints; you only need an mmproj clip file if you later point it at a vision model.

Troubleshooting

The recurring pain is llama-cpp-python itself: on Windows that shows up as "invalid wheel filename" during install or a missing llama.dll on import. Both are fixed by installing the package built for your backend (CUDA/Metal/ROCm) from the abetlen docs - the README calls this out explicitly. If the model loads but never uses your GPU, same cause. And if you're OOMing at the sampler step, you left unload off.