LLaVA (Managed Cache)

The classic LLaVA flow in this pack is a three-node dance: load the projector, load the GGUF, run the sampler. LLavaOptionalMemoryFreeSimple collapses that into a single node that loads both files itself, caches the model, and can free it the moment the run finishes. The "Optional Memory Free" in the name isn't marketing - it's the feature: turn on unload and the node hands the VRAM back after each queue instead of letting the model squat in memory.

What it's for

When you want LLaVA captioning in a graph without the loader boilerplate, or you're running a LLaVA node next to a big diffusion model and you need the VRAM back between runs. It's the same engine as the split loader path - GGUF + mmproj through llama.cpp - just self-contained.

How it works

It takes a ckpt_name (the GGUF) and a clip_name (the matching mmproj), both picked from your models/LLavacheckpoints folder, and builds a llama.cpp model handle internally. It keeps that handle cached per node instance so repeated queues are fast, and it keys the cache on every relevant setting - change any input and it rebuilds. The unload boolean is the memory-free switch: false keeps the model resident (fast repeats), true tears it down after each execution and requests a ComfyUI cache cleanup.

The inputs that matter

• ckpt_name / clip_name - the two files. The recurring warning applies: the projector must match the GGUF's model family, or you'll get noise.
• gpu_layers - how many layers to offload to the GPU. 27 is the default and works for 7B-class models; if it crashes or crawls, raise it (it defaults differently per build - -1 means "all" in the loader nodes).
• max_ctx - context window; 4096 default is plenty for single-shot captioning.
• image / prompt - the actual ask.
• temperature - 0.1 default; low is right for description work.
• unload - the namesake switch, explained above.

Output: a single STRING description.

Install

Standard for the pack - and remember this rides llama.cpp:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

If you're hitting llama-cpp-python install or GPU-offload problems, that's the pack's #1 support issue: install a wheel built for your backend (CUDA/Metal/ROCm) instead of the default CPU build.

Where it sits

Compared to the split-loading LLava Loader + Llava Clip Loader path, this is the "I just want a caption" option - fewer nodes, less graph ceremony, same output. The tradeoff is flexibility: it doesn't hand you a reusable model handle for other samplers, and the newer Advanced variant in the pack adds the full sampling controls if you outgrow this one. For a beginner who just wants a LLaVA caption node that doesn't hog memory, this is the one.