ComfyUI Node

✨ Auto-LLM-Text

Let a local LLM write the details you'd never type

By xlinx·Created 2 years ago·Updated 2 years ago· 32
✨ Auto-LLM-Text
  • clip
  • postive
  • negative
  • orignal-postive
  • orignal-negative
  • 🌀LLM-Text
  • 🌀LLM-Vision
  • 🌀postive+LLM-Text+LLM-Vision
llm_text_result_append_enabledtrue
text_prompt_postive1girl,
text_prompt_negative
llm_keep_your_prompt_aheadtrue
llm_recursive_usefalse
llm_apiurlhttp://localhost:1234/v1/chat/completions
llm_apikeylm-studio
llm_api_model_namellama3.1
llm_text_max_token50
llm_text_tempture0.30
llm_text_system_promptYou are an AI prompt word engineer. Use the provided keywords to create a beautiful composition. Only the prompt words are needed, not your feelings. Customize the style, scene, decoration, etc., and be as detailed as possible without endings.
llm_text_ur_promptA superstar on stage.
llm_before_action_cmd_feedback_type
llm_before_action_cmd
llm_post_action_cmd_feedback_type
llm_post_action_cmd

The name is doing a lot of work, but the trick is simple: you type one line - "a superstar on stage." - and a local LLM turns it into a full scene description that gets appended to your prompt before CLIP ever sees it. It's the ComfyUI half of xlinx's decadetw auto-prompt family (there's a matching sd-webui extension), and the pitch is "trigger more detail that you never thought." No cloud, no API key, no billing - it calls whatever OpenAI-compatible server you already run locally.

This is the node for batch generation where every image is a variation on a seed idea instead of a copy: type a story beat, get a different detailed composition each run. Prompt detail is the whole game for CLIP-era models, and a small LLM is a cheap machine to generate it.

How it works

Under the hood it's one HTTP POST per run. The node builds a chat-completions payload - your llm_text_system_prompt as the system message, your llm_text_ur_prompt as the user message - and sends it to llm_apiurl, which defaults to LM Studio's endpoint on port 1234. It reads choices[0].message.content back, appends it to your base text_prompt_postive, collapses double commas, and runs the result through your clip input into real CONDITIONING output. Your negative prompt is left alone - encoded as-is, the LLM never touches it.

The pack adds no real Python weight: requirements.txt lists only numpy, and the node uses requests and Pillow, which ComfyUI already ships. The real dependency is a running server - see Install below.

The inputs that matter

The node has a lot of widgets, and most have sane defaults. The ones a beginner actually touches:

  • llm_apiurl, llm_api_model_name - where to call and what model to ask for. Defaults are http://localhost:1234/v1/chat/completions and llama3.1; change the model name to match whatever you loaded in LM Studio or ollama.
  • llm_text_ur_prompt - your seed idea ("A superstar on stage."). This is the creative input.
  • llm_text_system_prompt - the persona. The default is a solid "AI prompt word engineer" that's tuned to output nothing but prompt text; leave it until you know why you're changing it.
  • llm_text_max_token - 50 by default, which gets you a short reply. Raise it (up to 1024) if you want the LLM to write longer passages.
  • llm_text_tempture - 0.3 by default. Yes, "tempture" - the author's typos are baked into the schema, along with orignal-postive and postive on the outputs.
  • llm_recursive_use - on, it chains the previous LLM reply into the next call so a batch becomes a continuing story. Fun, and prone to drifting into nonsense.
  • llm_keep_your_prompt_ahead - on by default; keeps your base tags ("1girl,") at the front of the assembled string.

Outputs and wiring

Three of the seven outputs are what you actually use:

  • postive and negative (CONDITIONING) - wire straight into your sampler's positive/negative inputs. The negative is exactly your text_prompt_negative; the positive is your base prompt plus the LLM's reply.
  • 🌀LLM-Text (STRING) - the raw LLM reply, handy for inspection.
  • 🌀postive+LLM-Text+LLM-Vision (STRING) - the fully assembled prompt as text, if you want to route it elsewhere. On this node 🌀LLM-Vision comes out empty; that output exists because the pack's other nodes share the same schema.

Install

Easiest via ComfyUI Manager: Custom Nodes Manager → search autoComfyUI-decadetw-auto-prompt-llm → install → restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/xlinx/ComfyUI-decadetw-auto-prompt-llm
# restart ComfyUI

Then the actual setup: install LM Studio (recommended, serves port 1234) or ollama (port 11434), and load a small text GGUF - the README's picks are Qwen1.5-4B-Chat Q2_K (under 2GB VRAM), Gemma 2 9B IQ2_M, or Llama-3-Taiwan-8B Q2_K.

Common issues

  • [Auto-LLM][Result][Missing LLM-Text]... appears in your prompt. The node doesn't fail loudly - when the server is unreachable it writes the error string into the output, which then gets encoded into your conditioning. If you see that text in the assembled prompt, your server isn't running or the URL/model name is wrong.
  • Replies too short. It's the 50-token default; raise llm_text_max_token.
  • VRAM. The LLM sits in VRAM next to your checkpoint. Keep the model small. On ollama, the README's suggested after-action curl (keep_alive: 0) unloads it between calls.
  • llm_before_action_cmd / llm_post_action_cmd run arbitrary shell commands. The README flags this as a security issue itself. Don't paste in commands you don't understand, and don't run this pack on a ComfyUI instance exposed to the internet - custom nodes execute with your full user permissions, which is exactly how the ComfyUI LLMVISION malware incident worked.
Category🧩 Auto-Prompt-LLM

Inputs (17)

NameTypeDefaultDescription
clipCLIP
llm_text_result_append_enabledBOOLEANtrue
text_prompt_postiveSTRING1girl,
text_prompt_negativeSTRING
llm_keep_your_prompt_aheadBOOLEANtrue
llm_recursive_useBOOLEANfalse
llm_apiurlSTRINGhttp://localhost:1234/v1/chat/completions
llm_apikeySTRINGlm-studio
llm_api_model_nameSTRINGllama3.1
llm_text_max_tokenINT5010–1024
llm_text_temptureFLOAT0.30-2–2
llm_text_system_promptSTRINGYou are an AI prompt word engineer. Use the provided keywords to create a beautiful composition. Only the prompt words are needed, not your feelings. Customize the style, scene, decoration, etc., and be as detailed as possible without endings.
llm_text_ur_promptSTRINGA superstar on stage.
llm_before_action_cmd_feedback_typeCOMBO4 options: Pass, just-call, LLM-USER-PROMPT, LLM-VISION-IMG_PATH
llm_before_action_cmdSTRING
llm_post_action_cmd_feedback_typeCOMBO4 options: Pass, just-call, LLM-USER-PROMPT, LLM-VISION-IMG_PATH
llm_post_action_cmdSTRING

Outputs (7)

NameTypeDescription
postiveCONDITIONING
negativeCONDITIONING
orignal-postiveSTRING
orignal-negativeSTRING
🌀LLM-TextSTRING
🌀LLM-VisionSTRING
🌀postive+LLM-Text+LLM-VisionSTRING