Nodes/comfyui-llm-assistant/Generate Stable Diffsution Prompt With LLM
ComfyUI Node

Generate Stable Diffsution Prompt With LLM

Have a chat model write your image prompt

By longgui0318·Created 2 years ago·Updated 2 years ago· 8
Generate Stable Diffsution Prompt With LLM
    • stable diffsution prompt
    prompt
    template_systemYou are an assistant bot.Follow the instructions and don't interpret the content too much.Be careful to answer in English.
    template_userI want you to act as a prompt generator for AI paintings. Your job is to provide detailed and creative descriptions that will inspire the AI to create unique and interesting images. You can choose one or more of [Lighting, Background, Art Style, Physical Characteristics, Behavior, Demeanor, Look, Period, Effects] to adjust or add details to the description to fit the theme as much as possible. When the scene description is complete, you can output "Scene Introduction:" and append your introduction to the tag. Please generate a prompt for this topic: {}
    stop
    response_pattern
    temperature1.00
    max_tokens300
    model_namenousresearch/nous-capybara-7b:free

    Yes, "Diffsution" is a typo baked right into the node's own name and output socket, not a typo in this article - worth knowing so you can find it in search and in your node list. Past that, this one does exactly what it says: you give it a rough topic, it asks a chat model to turn that into a detailed image-generation prompt, and hands you back a string ready to feed into your encoder. It's the "brainstorm buddy" node - useful when you know roughly what you want ("a lighthouse at night") but not the fifteen descriptive words that get a diffusion model to actually render it well.

    How it works, and where it fits

    Having an LLM write your image prompt for you isn't a novelty any more - it's become a mainstream step in a lot of ComfyUI workflows, and dedicated prompt-writing node packs draw real attention on release specifically because typing good prompts by hand is tedious. This node is a lightweight, DIY version of that idea baked into a general LLM-assistant pack rather than shipped as its own dedicated product.

    The mechanism is the same template-substitution trick as its sibling nodes: template_system sets the model's role ("act as an assistant... don't over-interpret... answer in English"), and template_user is a longer, pre-written instruction that tells the model to act as an AI-painting prompt generator, pick from a list of aspects - lighting, background, art style, physical characteristics, behavior, demeanor, look, period, effects - and weave them into a description. Your prompt input drops into that instruction's {} placeholder as the topic. Edit template_user if you want a different style of output - tag-heavy for an anime checkpoint, prose-heavy for a photoreal one, JSON-structured for models that read it well.

    One distinction worth being clear on: this is not the same thing as the LLM text encoders built into 2026-era checkpoints like Anima or Flux 2 Klein, where a language model is the encoder reading your prompt directly. This node runs before any of that - it's a separate helper step producing a plain text string. Whatever encoder your checkpoint actually uses reads this node's output exactly like it would read a prompt you typed by hand.

    Inputs and outputs that matter

    • prompt - your topic or rough idea, not a finished prompt. "A lighthouse at night" is a perfectly fine input.
    • template_user - the pre-written prompt-generation instruction; edit it to bias the style of what comes back.
    • model_name - a fixed dropdown of 21 chat models in provider/model-name form; the naming convention strongly suggests this pack calls OpenRouter, though the README doesn't name the provider directly.
    • temperature (0–1, default 1) - worth pushing toward the higher end here specifically, since more variety is usually what you want from a prompt generator, unlike the Translate node where you'd want it low.

    The output is a single string, unusually named stable diffsution prompt (typo and all) rather than the plain text its siblings use. Wire it straight into CLIPTextEncode.

    Installing it

    ComfyUI Manager: search comfyui-llm-assistant. Manually:

    cd ComfyUI/custom_nodes
    git clone https://github.com/longgui0318/comfyui-llm-assistant
    pip install -r requirements.txt
    

    Restart, then add a config.yaml with an API key - the README doesn't document the exact format, but given the OpenRouter-shaped model list, an OpenRouter key is the reasonable first thing to try.

    Common issues

    The default template_user explicitly tells the model to answer in English and to append a "Scene Introduction:" line after the tags - if you're generating prompts for a tag-based anime checkpoint, that trailing sentence is noise you'll want to strip or edit the template to drop, since most tag-trained models don't benefit from a prose sentence mixed into a tag list.

    Same two structural gotchas as the rest of this pack: a missing or wrong config.yaml fails at run time, not at load time, so a node that imports cleanly can still error on every execution until the key is right; and model_name is a closed 21-item dropdown rather than free text, so if a specific model starts failing (several in the list are older free-tier names that get retired), swap to another entry rather than assuming your setup is broken.

    CategoryLLM

    Inputs (8)

    NameTypeDefaultDescription
    promptSTRING
    template_systemSTRINGYou are an assistant bot.Follow the instructions and don't interpret the content too much.Be careful to answer in English.
    template_userSTRINGI want you to act as a prompt generator for AI paintings. Your job is to provide detailed and creative descriptions that will inspire the AI to create unique and interesting images. You can choose one or more of [Lighting, Background, Art Style, Physical Characteristics, Behavior, Demeanor, Look, Period, Effects] to adjust or add details to the description to fit the theme as much as possible. When the scene description is complete, you can output "Scene Introduction:" and append your introduction to the tag. Please generate a prompt for this topic: {}
    stopSTRING
    response_patternSTRING
    temperatureFLOAT1.000–1
    max_tokensINT300-1–2048
    model_nameCOMBOnousresearch/nous-capybara-7b:free21 options: nousresearch/nous-capybara-7b:free, mistralai/mistral-7b-instruct:free, gryphe/mythomist-7b:free, undi95/toppy-m-7b:free, openrouter/cinematika-7b:free, google/gemma-7b-it:free, +15

    Outputs (1)

    NameTypeDescription
    stable diffsution promptSTRING