Structured Prompt Generator

If you've ever watched a raw LLM output a prompt, you've seen the problem: "Sure! Here is a prompt for your image: A majestic..." - with the scaffolding, the fluff, the chat-bot preamble. LLavaPromptGenerator exists to kill that. It's the final stage of the pack's automatic-prompt pipeline: feed it keywords (or a description), and it returns only the prompt, structured as JSON, ready to wire into a CLIP Text Encode.

What it's for

The terminal node in the caption-to-prompt chain the pack is famous for: LLaVA describes an image → Get Keywords extracts the salient terms → this node builds a production-ready prompt from them. You can feed it the LLaVA output directly too, but the two-hop route (keywords first) gives it cleaner material to work from.

How it works

Like its sibling structured nodes, it runs a constrained chat against a model handle from LLMLoader: a system message ("Create one production-ready image-generation prompt and return only the requested JSON object"), a JSON schema, and a response-format constraint from llama.cpp so the model cannot wander into prose. The node then parses the JSON and returns the prompt field as a clean string. The mechanism is the same one the author shipped as the pack's structured-output feature - reliable extraction over hopeful prompting.

The inputs that matter

• prompt - keywords or a description of what you want.
• model - the CUSTOM handle. The author's guidance: works best with LLaVA 1.5 / 1.6 GGUFs.
• temperature - 0.15 default. Low for consistency; this node is about usable output, not surprises.

Output

A single STRING - just the prompt, no commentary. Wire it directly into a CLIP Text Encode and generate.

Install and gotchas

Standard pack install, standard llama.cpp caveat:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

Install llama-cpp-python with a wheel for your GPU backend or everything runs on CPU. And a fair warning on expectations: "works best with LLaVA 1.5/1.6" is doing real work there. Those models were trained on image-prompt pairs, so they naturally produce prompt-shaped text. If you load a generic chat GGUF and get prose, that's the model fighting the format - switch to a LLaVA-family GGUF and the node will behave the way the demo videos show.