Vision Prompt Helper

VisionPromptHelper does reverse prompting: reference image in, usable prompt out. Drop in a photo of a person and it returns either an edit instruction ready for an instruction-following editor (FLUX Kontext, Qwen Image Edit, FLUX.2) or a tight snippet describing one aspect of the image - face, hair, outfit, lighting. It's the node from comfyui-prompt-tools that turns your saved images into a vocabulary instead of a wall you stare at.

This is where the pack gets genuinely clever. Same shared engine selector as PromptHelper, but the mode dropdown splits into two families that do different jobs: edit modes produce a transformation instruction ("Change the background…", "preserve identity…") for the image-editing models, and describe modes produce a single-aspect snippet meant to be fed into the pack's PromptComposer. That split mirrors the "pipeline of specialists" pattern that's been winning in this space - chain narrowly-scoped steps instead of trusting one free-form rewrite, because a small model drifts less when its job is small.

How it works

Your image_1 tensor gets encoded to a base64 JPEG (resized so the longest edge hits max_edge, compressed to jpeg_quality) and sent alongside a mode-specific system prompt to the vision model. The default model, qwen3-vl:8b, is a genuine VLM - this is the one node in the pack where the model must actually be able to see. A text-only llama3.2 will happily take your image and return prose about it anyway, which is a useless failure mode to hit once before you remember why.

Outfit Transfer is the only mode that consumes image_2 (the outfit source; image 1 is the identity source). Every other mode silently ignores a second image even if you wire one in - the source code says so, so don't expect a "Background Change" to use two references. Edit-mode outputs are written for specific editors: the outfit prompt, for instance, is tuned for FLUX.2 multi-reference and demands explicit identity anchors ("the same woman from image 1" is not enough - it wants eye shape, hair, skin tone spelled out).

The inputs that matter

• intent - your instruction, e.g. "transfer the outfit from image 2 onto her". Can be left empty in describe modes.
• mode - the 15-way dropdown. Edit: Outfit Transfer, Hair Change, Body Reshape, Background Change, Pose Change, Combined Edit. Describe: Picture, Face, Hair, Body, Pose, Outfit, Background, Lighting, Composition.
• image_1 - required IMAGE input; image_2 only for Outfit Transfer.
• engine / base_url / model - pick a vision-capable model. Ollama's qwen3-vl:8b (the default) or a qwen3-vl:32b if your card can hold it.
• jpeg_quality (default 85) and max_edge (default 1280) - defaults are fine; drop max_edge if you want faster calls or the model is misreading fine detail.

Two outputs: vision_prompt (the generated prompt) and debug_info (a string with mode/engine/latency that you'd normally wire to a ShowText node). Note the error handling: problems don't crash the graph - they come back as an ERROR: … string in vision_prompt with the reason in debug_info. If you're staring at a literal "ERROR:" on your ShowText, check debug_info for whether the engine resolution or the image encoding failed.

Installing

Same pack, same install: ComfyUI Manager search comfyui-prompt-tools, or clone into custom_nodes. The heavy lift isn't the node, it's the model - you need a vision-capable LLM running:

cd /path/to/ComfyUI/custom_nodes
git clone [email protected]:wernerberHH/comfyui-prompt-tools.git
# optional but recommended: enables URL/model autocomplete dropdowns
pip install pyyaml
ollama pull qwen3-vl:8b

Restart ComfyUI. The pyyaml dependency just powers the dropdowns; skip it and the URL/model fields degrade to plain text.

Common issues

The classic three: running a text-only model (it can't see - switch to a VLM), forgetting image_2 in Outfit Transfer (you get an explicit "requires a second reference image" error), and expecting describe modes to be faithful. Every VLM muddles multi-subject attribution - who's wearing what in a two-person frame is exactly where these models fumble, so audit describe output before you build a whole scene on it. And watch the latency: vision calls are slower than text ones, and the whole image goes over the wire each run. If you're on a local Ollama it stays on your box; pick OpenAI/Gemini as the engine and that reference image is now leaving your machine, so know which one you chose.