Grok Flux Prompt Optimizer

You've got an image you like - maybe it's your own render, maybe it's a reference - and you want a Flux prompt that recreates its vibe with a twist. That's a vision-language task, and the Grok Flux Prompt Optimizer is a node built exactly for it: you feed it an image, tell it what to change, and it returns an optimized Flux prompt. It's from the babydjacNODES pack and, as the name says, it's a Grok/API node - so it needs an xAI key and costs per call.

This fits the well-trodden "VLM reads an image, an LLM writes the prompt" pattern the community converged on (llm-in-comfyui.md). You're chaining a vision model to a prompt writer in one node instead of two.

How it works

The node converts your image tensor to a JPEG, downscales it if it's over ~1 megapixel (so big renders don't blow up the payload), base64-encodes it, and sends it to Grok's vision endpoint along with your user_instruction. The prompt asks for a Flux-optimized prompt back - one that matches the reference image but applies your requested edits, tuned by style_preference and detail_level.

Two implementation details worth knowing:

• It uses the first image in the batch only (image[0]), so a multi-image input just uses frame one.
• It implements IS_CHANGED, hashing the image shape plus the text inputs - meaning the node re-runs when you change the image or instruction, but doesn't spam the API on unrelated re-queues. On the flip side, hashing the shape means two different images of the same resolution might not trigger a rerun. If the output feels stale after swapping images, that's why.

The inputs that matter

• image - required. Any IMAGE tensor; usually a Load Image node.
• user_instruction - required. "Make it more vibrant and colorful" is the default, and it's a good template.
• api_key - required unless you set XAI_API_KEY or GROK_API_KEY in the environment; this node does fall back to env vars, unlike some siblings in the pack.
• style_preference - photorealistic / artistic / cinematic / portrait / landscape / abstract / anime / concept_art.
• detail_level - basic / detailed / very_detailed.

Output is a single STRING, optimized_prompt, ready for a Flux text encoder. It's not an output node, so you can chain it into anything.

Install

ComfyUI Manager → babydjacNODES, or git clone https://github.com/babydjac/babydjacNODES into ComfyUI/custom_nodes, restart, hard-refresh. No extra pip deps.

Troubleshooting

• "[Error] Missing API key" - fill the widget or set XAI_API_KEY/GROK_API_KEY as an environment variable.
• Stale output after changing the reference - the IS_CHANGED hash uses image shape, not content. Change the instruction text slightly (or the resolution) to force a re-run, or clear the cache.
• Cost creep - every run is a paid vision call. Vision tokens cost more than text, so use this for deliberate iterations, not batch sweeps.

The realistic use: keep a reference in a Load Image node, iterate the instruction, and let the optimized prompt feed your sampler. It's a genuinely handy workflow - just remember you're renting Grok's eyes for it.