IF Image to Prompt🖼️

IF Image to Prompt is the reverse-engineering node of the IF_AI_tools pack. You drop in an image, it asks a vision LLM to describe it, and the response output comes back as a ready-to-paste Stable Diffusion prompt. It's the fastest way to grab the "vibe" of a reference image - a style you found on Civitai, a screenshot, a mood board - without hand-writing keywords. Wire the response straight into a CLIP Text Encode and you're generating variations within a minute.

The pack name tells you the design lineage: this is the same family as IF Prompt Maker, sharing the identical input schema. Both are being migrated to the newer ComfyUI-IF_AI_PromptImaGen repo, so treat them as the mature, stable versions of that idea.

How it works

The node takes your images, converts them for API/local consumption (the pack has a convert_images_for_api helper for this), and sends them to the LLM with a system prompt that instructs it to write SD-style prompts. Which system prompt it uses is up to you: the profiles dropdown pulls from JSON files in the pack's IF_AI/presets folder - IF_PromptMKR, IF_Omost, IF_HunyuanVideoPrompt, and ~27 others, all editable if you want your own.

The strategy input changes the whole behavior:

• normal - describe the image as a prompt (the default).
• omost - use the Omost tool to produce structured canvas-conditioning layout data instead of plain text.
• create / edit / variations - switch to generating images via the OpenAI image API, which is why batch_count exists.

On top of that you get the pack's preset stacks: embellish_prompt, style_prompt, and neg_prompt (63 styles, 29 embellishments, 24 negatives) which get glued onto the LLM's response. The auto / auto_mode toggles batch up to four images at a time through combo/mix prompt modes.

The inputs that matter

• images - the reference image(s). This is the whole point.
• llm_provider / llm_model - ollama + a vision model like llama3.2-vision is the free path; the cloud providers need their env keys.
• user_prompt - an extra instruction, e.g. "describe the lighting and composition in detail."
• profiles - the system-prompt preset that shapes the output.

Outputs: response (the prompt - feed to CLIP Text Encode), negative (if a negative preset is active), question (echo), omni (tool payload for omost strategy), generated_images and mask (for the create/edit/variations strategies).

Installation

ComfyUI Manager, search "IF_AI_tools". Or:

cd ComfyUI/custom_nodes
git clone https://github.com/if-ai/ComfyUI-IF_AI_tools.git

Then pip install -r requirements.txt (a long list - the RAG stack like byaldi and sentence-transformers is included even though this node doesn't strictly need it), and make sure you have Ollama running with a vision model if you're going local.

Gotchas

The obvious trap: your chosen model must actually be a vision model. Ollama will cheerfully accept llama3.2 (text-only) and return garbage or an error when you feed it images - ollama run llama3.2-vision first. And remember the repo is archived: the author moved on to comfy-deploy's LLM toolkit, so don't expect new features here. If all you want is image-to-prompt, the current recommendation is the successor PromptImaGen node, which requires you to disable this pack first - the two will collide.