VLM Prompt Refiner (kppb)

Every other node in this pack builds prompts from dropdowns. This one is different: VLM Prompt Refiner (kppb) looks at an actual image and writes the prompt for you. You feed it a character reference photo, it extracts the identity, merges that with your scene settings, and hands back a complete Klein-ready prompt. It's the "have an LLM translate what you want into a well-structured prompt" trick the community converged on for LLM-encoded models, but packaged as a node that can see the pixels.

How it works

The node has two backends and they share one pipeline: your reference images get base64-encoded, a system prompt is assembled per mode, and either Ollama or the Claude Code CLI is called with both the text and the images.

• Ollama (default) - direct HTTP calls to Ollama's REST API, no extra custom-node dependencies. The default model is huihui_ai/qwen3-vl-abliterated:32b-instruct-q8_0, and it auto-pulls it on first use if it's missing:

  ollama pull huihui_ai/qwen3-vl-abliterated:32b-instruct-q8_0

• Claude Code CLI - toggles use_claude_code, picks claude_model (opus, sonnet, haiku), and the node shells out to the claude binary. Nice when you don't want a 30GB+ VLM competing with Klein for VRAM.

The modes

• describe & enhance - the default. It takes your prompt_json from the Prompt Builder, merges character identity from the reference image with your scene settings, and writes one optimized paragraph.
• image edit aware - feed it edit_prompt from the Image Edit Composer and it describes the final scene after the edits, which is what an image-edit subgraph actually needs.
• caption only - just describes the reference image as a Klein 9B generation prompt.
• dataset generation - writes LoRA training captions with your trigger_word (default ohwx) as a placeholder. That's a genuinely useful mode if you're building a dataset and want caption consistency.

Outputs are refined_prompt (the thing you condition on), image_caption, filename_prefix, and - if you flip generate_video_prompt - a Wan 2.6 I2V motion prompt.

The inputs that matter

• character_ref - required. The identity source. No image, no output.
• ollama_url / model - point at your Ollama instance.
• mode - pick what you want it to produce.
• temperature (default 0.3) and seed - lower temperature for focused output; set a seed for reproducibility.
• positive_prompt / prompt_json / edit_prompt - connect these from the Prompt Builder so the VLM knows your exact settings instead of guessing. positive_prompt doubles as the fallback if the VLM returns empty.
• unload_model (default on) - offloads Ollama from VRAM after inference. Leave it on for localhost (Klein needs that VRAM) and turn it off on a remote/RunPod box where reloading between iterations is the slow part.

Installing and gotchas

Install is the pack standard - ComfyUI Manager search "Photoreal Prompt Builder", or git clone into custom_nodes/, then restart. The README claims no pip dependencies, with one asterisk: this node needs numpy and Pillow, and the pack silently hides the node if they're missing (both ship with any normal ComfyUI install, so it usually just works). Then:

# Ollama backend
curl -fsSL https://ollama.com/install.sh | sh   # or install from ollama.com
ollama pull huihui_ai/qwen3-vl-abliterated:32b-instruct-q8_0

# Claude backend instead
npm install -g @anthropic-ai/claude-code

Where people get burned: running Ollama on the same GPU as Klein and wondering why generation got slow - that's the unload_model: true fix, right there in the tooltip. And if you're using a hosted/API model with content filters, flip sfw_prompt on; it softens the output so it survives the filter instead of bouncing.