EmAySee_Qwen / OpenAI Prompt From Image

EmAySee QwenPromptFromImage is the pack's heavyweight: a vision-language node that looks at an image and writes a rich text-to-image prompt for it. It runs Qwen2.5-VL or Qwen3-VL models locally through transformers, or - if you don't want to host weights - it can call any OpenAI-compatible vision endpoint, from a local Ollama server to OpenRouter. One node, two backends, and a genuinely useful output: a caption you can pipe straight into a CLIP Text Encode.

The image-to-prompt loop is one of the most practical things in local generation. You see an image you like, you want a prompt that captures it, and a Qwen-VL model is very good at that job - the Qwen2.5-VL line is one of the most-cited local VLM routes in the ComfyUI community precisely because it's Apache-licensed, runs at useful sizes (2B to 8B), and writes clean prompts when you prompt it right. This node wraps that whole flow: load the model, feed the image, enforce a system prompt, and clean the output into a single line.

How it works

Two paths, chosen by the backend dropdown.

Local: the image tensor is converted to a PIL image and handed to a Qwen-VL model loaded via HuggingFace transformers - Qwen/Qwen2.5-VL-3B/7B-Instruct or the newer Qwen3-VL 2B/8B/235B lines. You pick the model, the device, quantization (bf16, fp16, int4_bnb, int8_bnb, fp32_cpu), and whether to use flash attention. The model generates, and the output is split at the </think> tag so reasoning models give you thinking_text separately from the final prompt_text. The node caches the loaded model and - by default - unloads it after each run to free VRAM.

OpenAI-compatible: the image is base64-encoded and POSTed to {base_url}/v1/chat/completions as an image_url message. The default base URL is http://127.0.0.1:11434 - Ollama's local endpoint - so if you already run Ollama with a vision model, this backend is zero-setup. It also works against OpenRouter (there's even an openrouter_providers field to pin providers) or any compatible gateway. verbose_output prints a nice token/cost breakdown to your console when you want it.

The inputs that matter

The full list is long; these are the ones you'll actually touch:

• image - the IMAGE to caption.
• backend - local or openai_compatible.
• qwen_model - the model ID dropdown; only relevant for local. If you have a local path, local_model_path overrides it.
• system_prompt - the instructions. The default ("convert the given image into a rich, image-generation prompt... return ONLY the final prompt as a single line") is genuinely good - leave it until you know why you're changing it.
• max_new_tokens and temperature - generation controls; temperature 0.2 default is right for captioning.
• safe_quants - if on, the node silently upgrades your quant choice to what the device can actually run (e.g. fp32_cpu on CPU), which prevents a lot of "it loaded but errors" pain.

Outputs: prompt_text (the cleaned single-line prompt) and thinking_text (reasoning, empty for non-thinking models).

Installing it

Same pack install as everything else:

# ComfyUI Manager: search "ComfyUI_EmAySee_CustomNodes" and install
# or manually:
cd ComfyUI/custom_nodes
git clone https://github.com/EmAySee/ComfyUI_EmAySee_CustomNodes

Then restart ComfyUI. This node is the one with real dependencies. The pack ships no requirements.txt, so for the local backend you must install transformers (recent enough for Qwen3-VL), bitsandbytes for the int4/int8 quants, and requests yourself:

pip install transformers accelerate bitsandbytes requests

First local run downloads the model from HuggingFace - a 2B model is a few GB, 8B is bigger - so budget for that. The OpenAI-compatible backend needs only requests.

Common issues

Local loading is where most people hit walls. If transformers is missing or old, the node raises a clear "transformers is missing or too old" error - install it fresh. Bits-and-bytes quant options require a CUDA device; on CPU, safe_quants will force you to fp32_cpu, which is slow but works. And a 235B model through this node is a "do you have a big server" option, not a home-GPU one. On the API side, make sure your base URL is reachable and your model accepts image_url messages - not every "OpenAI-compatible" server implements vision. The author's README is AI-generated and doesn't document this node properly, and the category (d3cker/Prompt-Generator) is a leftover from where it was developed - you'll find it there, not under an EmAySee heading.