🐾Prompt_Generator

If you're building a Flux LoRA dataset, you've already learned that captioning is the single highest-leverage thing you'll do - and the most tedious. Flux was trained on natural-language descriptions, not comma-separated tags, so your captions should read like sentences a person wrote. That's exactly the gap this node fills: drop an image in, get a flowing natural-language prompt back, all on your own GPU, with no API key and no cloud account.

Prompt_Generator is the main node in the pzc163/Comfyui_MiniCPMv2_6-prompt-generator pack, and it's a thin ComfyUI wrapper around a HuggingFace model of the same name: a fine-tune of OpenBMB's int4-quantized MiniCPM-V 2.6, trained on 3,000+ Midjourney image/prompt pairs (on a pair of 4090s, per the author's README). It's a vision-language model aimed squarely at turning images into the descriptive, Midjourney-flavored prompts people actually prompt with.

How it works

On its first execution the node downloads the model into ComfyUI/models/LLM/MiniCPMv2_6-prompt-generator - about 6 GB, and you don't have to trigger anything; it just happens. The int4-quantized model loads through bitsandbytes (roughly 7 GB of VRAM, per the README) onto your ComfyUI device. Each caption pass runs as beam search with sampling off, which keeps output stable enough for training data rather than hallucinating something different every run.

The caption_method dropdown picks the style:

• caption - a detailed description plus a list of generation-style tags.
• short_prompt - a compact, Stable Diffusion-ready prompt.
• long_prompt - the default; a rich Midjourney-style description capped around 256 tokens.

That last one is where you'll live for Flux datasets.

The inputs that matter

You set exactly four things: folder_path, caption_method, max_new_tokens (default 2048, max 4096), and num_beams (default 3). For a single image, ignore folder_path entirely and wire a LoadImage into the images input instead. The optional prefix_caption and suffix_caption strings get glued around whatever the model writes, and replace_tags lets you do search:replace pairs (separated by semicolons) on the output - handy for normalizing a trigger token.

The outputs are images, filenames, captions, folder_path, and batch_size; the first three come out as lists. Wire captions into a ShowText to eyeball results, or straight into a CLIPTextEncode if you want to regenerate the image from the fresh prompt.

Where people trip

First: the model loads from disk on every single execution - there's no caching in the node. Every queue run starts with a multi-GB model load before any captioning happens. Expect a real wait at the start of each run and batch your folder processing accordingly.

Second, replace_tags has a guard: if the field still contains the eg: placeholder text, no replacement happens. Clear it first.

Third, the optional captions input doesn't replace anything - whatever you feed it gets prepended to the output. That's the hook for the author's "concatenate description with WD14 tags" workflow: run a tagger, feed its tags in here, and get caption+tags on one line, which is a strong combo for training.

Install and first run

Via ComfyUI Manager, search for MiniCPMv2_6-prompt-generator. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/pzc163/Comfyui_MiniCPMv2_6-prompt-generator
cd Comfyui_MiniCPMv2_6-prompt-generator
pip install -r requirements.txt

The requirements pull in transformers, bitsandbytes (needs a CUDA build; on Windows this is the dependency most likely to fight you), accelerate, decord, and sentencepiece. Restart ComfyUI and run the workflow - the first run downloads the model, so give it a minute.

Honest verdict: JoyCaption remains the community default for this job and it's excellent. But this is a fully local, Midjourney-flavored alternative, which is nice when you want captions that match how you'll actually prompt at inference, or you just want everything on your own box. For Illustrious/Pony-style datasets, skip this - there you want WD14 tags, not natural language.