Loader Text to Prompt Model 🐼

What it is

The loader for the Text to Prompt node's Qwen-family models. This is the pack's other prompt-rewriting path - separate from the T5/superprompt one - and its default model is worth knowing about specifically, because the README talks about it directly: hahahafofo/Qwen-1_8B-Stable-Diffusion-Prompt, the pack author's own fine-tune of a small Qwen chat model, trained on 35,000 examples specifically for turning rough ideas into Stable Diffusion prompts. The README's own framing: it's cheap to run and "runs at a considerable speed on CPUs" - which is exactly why this loader defaults to device: cpu rather than cuda, unlike almost every other loader in this pack.

How it works

Standard loader shape: pick a model, pick a device, decide on memory tradeoffs, get back a TEXT2PROMPT_MODEL that the Text to Prompt node uses for generation.

Inputs and outputs

• model - five choices, defaulting to hahahafofo/Qwen-1_8B-Stable-Diffusion-Prompt. The other four are stock Qwen1.5 chat checkpoints at increasing sizes: Qwen/Qwen1.5-0.5B-Chat, Qwen1.5-1.8B-Chat, Qwen1.5-4B-Chat, and Qwen1.5-7B-Chat. The default is purpose-tuned for prompt rewriting specifically; the stock Qwen models are general-purpose chat models that'll follow the system prompt and prefix instructions reasonably well but weren't fine-tuned for this exact task. Bigger stock models will generally follow instructions more reliably at the cost of speed and memory - reach for Qwen1.5-7B-Chat if the 1.8B default is giving you flat or generic rewrites and you have the hardware to spare.
• device - defaults to cpu, unusually for this pack. That default only really makes sense for the small default model; if you switch to Qwen1.5-4B-Chat or Qwen1.5-7B-Chat, switch this to cuda too or expect generation to crawl.
• low_memory - defaults to true. Turn it off if you have VRAM to spare and want faster generation over a smaller footprint.

Output: TEXT2PROMPT_MODEL, wired into Text to Prompt.

Installing it

Ships as part of the full pack:

cd ComfyUI/custom_nodes
git clone https://github.com/zhongpei/Comfyui_image2prompt

Or install via ComfyUI Manager, searching "Comfyui_image2prompt." (The README's own clone command points at a hyphenated URL that 404s - use the underscored one above.) Restart ComfyUI afterward.

Model weights aren't bundled - they pull from Hugging Face on first use. If auto-download doesn't fire, the README's manual path (huggingface-cli download, with HF_ENDPOINT=https://hf-mirror.com for users behind the Great Firewall) applies to these models the same way it does the vision-language ones.

Common issues

Leaving device on cpu after switching to a bigger model. The default combination (small model, CPU) is fine. Swapping to Qwen1.5-7B-Chat while leaving device: cpu is where people get burned - that's a 7B chat model trying to generate on CPU, and it will be dramatically slower than they expect. Change both together.

Expecting the stock Qwen models to write Stable Diffusion prompts as well as the fine-tuned default. They're general chat models, not tuned for this task specifically - you'll usually get better prompt-shaped output from hahahafofo/Qwen-1_8B-Stable-Diffusion-Prompt at the same or smaller size than from an untuned Qwen1.5 checkpoint, at least without a much more carefully written system_prompt and prefix on the Text to Prompt node to compensate.

This loader alone produces nothing usable - it needs to be wired into Text to Prompt, which is where the actual generation controls (temperature, max tokens, system prompt) live.