Nodes/Comfyui_CXH_joy_caption/CXH_Min2_6_prompt_Run
ComfyUI Node

CXH_Min2_6_prompt_Run

Turn any image into a Stable Diffusion prompt with MiniCPM-V

By StartHua·Created 2 years ago·Updated 2 years ago· 624
CXH_Min2_6_prompt_Run
  • pipe
  • image
  • STRING
promptProvide a detailed description of the details and content contained in the image, and generate a short prompt that can be used for image generation tasks in Stable Diffusion,remind you only need respons prompt itself and no other information.
max_tokens1024
temperature0.70
seed656545

CXH_Min2_6_prompt_Run is the single-image inference node for the pack's MiniCPM-V 2.6 prompt generator. Feed it an image and it returns one thing: a generation-ready Stable Diffusion prompt describing what's in the frame. The default prompt input spells out the whole philosophy - "Provide a detailed description of the details and content contained in the image, and generate a short prompt that can be used for image generation tasks in Stable Diffusion, remind you only need respond prompt itself and no other information." - which is exactly what makes it different from the JoyCaption nodes: MiniCPM is told to produce a usable prompt, not a caption.

Where JoyCaption writes natural-language descriptions (great for training datasets), this node writes SD-style prompts with the noise stripped out. That makes it a natural fit for reverse-prompting: load an image, get a prompt, feed it to a KSampler to re-create or riff on the image. The README positions it as the middle-speed option - faster than JoyCaption, slower than Florence 2.

How it works

It's a thin wrapper over MiniCPM-V's built-in chat API. Your image gets converted from a ComfyUI tensor to PIL, bundled with the prompt text as a user message, and passed to pipe.text_model.chat(...) with sampling=False, stream=False, plus your max_tokens and temperature. Whatever the model returns is your output string. There's a seed input, but the source doesn't actually use it for sampling decisions - treat it as carried-over plumbing.

Inputs and output

  • pipe - from CXH_HG_Model_Load.
  • image - any IMAGE tensor.
  • prompt - multiline; the default is the SD-prompt instruction above. You can swap in your own instruction (e.g. "describe in five words" or "make it a MidJourney prompt") - this model follows instructions well.
  • max_tokens (10–4048, default 1024), temperature (0–1, default 0.7), seed.
  • Output: one STRING - the generated prompt.

Wire the string into your prompt encoder / KSampler, a Show Text node, or anywhere a prompt goes.

Installing

Same pack install - ComfyUI Manager (search "Comfyui_CXH_joy_caption") or:

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

The model download happens on the loader node (CXH_HG_Model_Load) first run.

Troubleshooting

  • Slow first inference is just the model loading. After that it's one of the faster options in the pack.
  • Prompt comes back as an essay even though you asked for a short prompt - MiniCPM-V sometimes over-explains. Crank max_tokens down or strengthen the instruction text.
  • The output is only a prompt, not a caption. If you're captioning a training set, use the JoyCaption or Florence 2 nodes instead; this one is for prompt extraction.
  • For batch prompt generation you'd loop this with a workflow-level iterator - the pack's own batch nodes target captioning, not this one.
CategoryCXH/LLM

Inputs (6)

NameTypeDefaultDescription
pipeCXH_Hg_Pipe
imageIMAGE
promptSTRINGProvide a detailed description of the details and content contained in the image, and generate a short prompt that can be used for image generation tasks in Stable Diffusion,remind you only need respons prompt itself and no other information.
max_tokensINT102410–4048
temperatureFLOAT0.700–1
seedINT6565450–1000000

Outputs (1)

NameTypeDescription
STRINGSTRING