GR Prompt Generator

This one's a real vision-language model wired into a node, not a text-template trick. Feed it an image and a question - the default is literally "Describe this image in detail" - and it runs an actual VLM to answer, handing back a text prompt you can use for img2img, ControlNet reference, dataset captioning, or just understanding what a model saw. It's the difference between guessing at a prompt for an existing image and having something actually look at it and tell you.

How it works

You pick a model from a list that spans a real range of sizes and vendors: Qwen2-VL (2B/7B), Qwen2.5-VL (3B/7B/72B-Instruct), Moondream (moondream1/moondream2), and HuggingFace's SmolVLM family (SmolVLM-Instruct, SmolVLM-256M, SmolVLM-500M). That range matters - Moondream and the small SmolVLM variants are built to run fast on modest hardware, while Qwen2.5-VL-72B is a genuinely large model that needs serious VRAM and a long first-run download. Pick based on what your machine can actually hold in memory, not just which one sounds most capable.

The node sends your question to the model alongside the image, optionally prepends pre_text to the result (useful for forcing a consistent prefix across a batch of captions, like always starting with a style tag), and samples the model's answer using temperature and max_new_tokens the way any LLM generation call would - higher temperature gives you more varied phrasing across runs, more tokens lets it write a longer description before it's cut off.

The inputs and outputs that matter

• question (STRING, multiline, default "Describe this image in detail.") - what you're asking the model. Change this to steer what it focuses on - "list the clothing and colors in this image" gets you a very different result than the default.
• image (IMAGE, optional) - what it's looking at. Technically optional in the schema, but without it you're just prompting a language model with no visual grounding.
• model - pick based on your hardware; smaller Moondream/SmolVLM variants are the sane default unless you specifically need Qwen2.5-VL's stronger detail.
• keep_model_loaded (BOOLEAN) - turn this on if you're running this node repeatedly in a session; it skips reloading the model each time at the cost of holding it in VRAM/RAM between runs.
• prompt (STRING, output) - the generated description, ready to wire into a CLIPTextEncode or a text-preview node.

How to install it

Via ComfyUI Manager: search "GraftingRayman", install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/GraftingRayman/ComfyUI_GraftingRayman

then restart. This pack shares one dependency across every node, this one included - OpenAI's CLIP package, installed separately from ComfyUI's own:

# portable build
.\python_embeded\python.exe -m pip install git+https://github.com/openai/CLIP.git

# system python
pip install git+https://github.com/openai/CLIP.git

On top of that pack-wide requirement, this specific node will download the VLM weights for whichever model you select the first time you run it - expect anywhere from a few hundred MB (SmolVLM-256M) to tens of GB (Qwen2.5-VL-72B) depending on which size you pick, and budget the disk space and first-run wait accordingly.

Common issues & troubleshooting

Node missing from search. Check the CLIP install above first - a pack-wide requirement, unrelated to the VLM itself.

First run hangs or takes forever. That's the model download, not a crash - check your console for download progress before assuming something's broken, especially on the larger Qwen2.5-VL variants.

Out of memory on a large model. This is the most common failure with the 72B Qwen2.5-VL option specifically - it's not a realistic pick on consumer VRAM. Drop to the 7B or 3B variant, or to Moondream/SmolVLM, if you're hitting OOM errors.

Descriptions are too generic or miss what you care about. Tune question rather than fighting the model's default framing - asking specifically for what you need (lighting, composition, clothing, mood) gets noticeably more useful output than the generic "describe this image" default.