llava

This is the coolest node in the pack, and the one that most deserves to have survived. The llava node is a local vision-language model wired straight into the graph: you feed it an IMAGE, it looks at it, and it returns a TEXT caption. No cloud API, no key, no server. It's what made the author's "positive feedback loop" demo possible - the LLM reasons about an image, which means the diffusion model and the language model can finally talk about what they can see.

Mechanically it's llama.cpp running LLaVA (the vision-capable model family) with a separate clip projector for the image side. That's why it takes two model files, not one. It's the same GGUF-loading approach as the pack's llama-cpp node, just with vision attached.

What you set

• image - the IMAGE to look at. Any image in the graph: a preview, a generated frame, an upscale pass.
• Model - default ggml-model-q4_k.gguf, the vision model itself.
• Clip_Model - default mmproj-model-f16.gguf, the vision-projector file. Both come from the exact source the README names: https://huggingface.co/mys/ggml_llava-v1.5-7b. Grab both files, put them in the pack's models folder.
• system_message - default: "You are an assistant who perfectly describes images." This is the persona; change it to steer what the caption emphasizes.
• Prompt - default: "Describe this image in detail please." The actual instruction, per image.
• models_path - optional override for where it looks for the GGUF files.
• n_ctx - 2048 default, same story as llama-cpp.
• n_gpu_layers - default 0, and this is the one to touch: it's how many layers to offload to your GPU. On a GPU box, raise it and watch speed improve dramatically. On CPU, leave it.
• temp - default 0.1, nicely low for captioning where you want consistency.

Output: TEXT, which you can DisplayString, or feed into Conditioning and let the caption drive a re-generation.

Where people get burned

The default n_gpu_layers = 0 means CPU-only out of the box, and a 7B vision model on CPU is not quick. If you have a GPU and it feels slow, the fix is right there in the widget. Also: the model files are large and named exactly as the defaults expect - miss the filenames and the node fails at load with a file-not-found that can look like a crash. And while LLaVA 1.5-7B was great in late 2023, it's an old model now; if you're building something serious today you'd reach for a newer VLM. The pack itself is also a fossil: xXAdonesXx/NodeGPT 404s as of this writing and the author's account is gone, so the README's clone install below is dead and ComfyUI Manager won't find it. The surviving fork (Guizmus/NodeGPT) only carries the pack's earliest build - no llava.

cd ComfyUI/custom_nodes
git clone https://github.com/xXAdonesXx/NodeGPT

Restart to auto-install requirements. If the concept appeals - and it should - plenty of maintained vision nodes do this today, but this one is still the cleanest illustration of "image in, words out, all local" you'll find in the pack.