ComfyUI Node

Media Describe

The node that turns any image or video into a structured Wan-ready prompt

By sammykumar·Created 12 months ago·Updated 8 months ago· 6
Media Describe
  • llm_studio_options
  • overrides
  • all_media_describe_data
  • raw_llm_json
  • positive_prompt_json
  • positive_prompt
  • prompt_request
  • height
  • width
media_processed_path
use_custom_promptsfalse
custom_system_promptGenerate a Wan 2.2 optimized text to image prompt. You are an expert assistant specialized in analyzing and verbalizing input media for instagram-quality posts using the Wan 2.2 Text to Image workflow. DECISIVENESS REQUIREMENT: Always provide definitive, certain descriptions. When you see something that could be described multiple ways, make a confident choice and state it as fact. Never use uncertain language like "appears to be", "seems to be", "might be", "possibly", "likely", or "or". Never mention watermarks, logos, branding, or any textual overlays. Return **only** a single valid JSON object (no code fences, no extra text) with **exactly five** string fields in this exact order: 1. "subject" - Detailed description of the main subject 2. "clothing" - Clothing and style details 3. "movement" - Pose, gesture, or implied motion 4. "scene" - Setting, environment, and background elements 5. "visual_style" - Combined lighting, camera details, rendering cues, mood/genre descriptors, and overall aesthetic direction Each field's value is one fully formed paragraph (a single string) for that category.
custom_user_promptPlease analyze this image and provide a detailed description in the JSON format specified in the system prompt.

Media Describe is the center of gravity of this pack - the node everything else orbits. Feed it a path to an image or video and it sends the media to a vision model (Gemini by default, or your local LM Studio) with a strict system prompt, then returns a structured description broken into five named paragraphs: subject, clothing, movement, scene, and visual style. Those five paragraphs are the whole design philosophy: separate concerns so you can override one without regenerating the rest.

The required input is a single string, media_processed_path - "path to the processed media file from Media Selection node." That's your cue for how the pack intends this to be fed: MediaSelection picks the media, optionally trims and resizes it, and hands you a clean path. You can also just type a path to an image or video file directly.

The optional inputs are where the power lives. llm_studio_options switches the backend from Gemini to your local LM Studio (the pack's local-first play - same output, no cloud, no key). overrides accepts an object from Media Describe - Overrides, letting you hand-tune individual paragraphs before they're assembled. use_custom_prompts flips it into manual mode so you can send your own custom_system_prompt and custom_user_prompt instead of the auto-generated ones - the defaults baked in are already fine (a Wan 2.2-optimized five-field JSON instruction set), but when you want total control, this is the switch.

Seven outputs, and you'll use most of them. all_media_describe_data is the full structured payload - the thing you feed Control Panel Overview to audit. raw_llm_json is the unprocessed model output. positive_prompt_json is the five-field structure as JSON (feed this to Control Panel Prompt Breakdown). positive_prompt is that same content flattened into one text prompt - the actual string you'd feed a CLIP text encoder or T2I model. prompt_request is a log of what was sent. And height/width are INTs - the inferred output dimensions (the node defaults to 832×480 for landscape, 480×832 for portrait), so your downstream resize/empty-latent nodes can stay in sync automatically.

Mechanically it's a Gemini call with retry logic (three attempts, five-second backoff for the ever-popular 429 overload) behind a media cache, so re-running with the same media can skip the API call. The prompt_style knob - set in GeminiUtilOptions or the LLM Studio options - switches between descriptive Text2Image output and single-sentence ImageEdit instructions, so the same node feeds both generation and editing workflows.

Installing

Part of Swiss Army Knife:

  • ComfyUI Manager: search "Swiss Army Knife", install, restart.
  • Manual:
cd ComfyUI/custom_nodes
git clone https://github.com/sammykumar/ComfyUI-SwissArmyKnife.git
cd ComfyUI-SwissArmyKnife
pip install -e .

The workflow it's built for

Media Selection → Media Describe → (Media Describe - Overrides →) Control Panel breakdown → wire positive_prompt into your Wan 2.2 T2I/VACE pipeline, with height/width driving your latent sizes. That's the pack's canonical loop, and it's a genuinely nice one: describe media once, turn it into a structured prompt, iterate on individual paragraphs instead of the whole thing.

Gotchas

  • Gemini needs a key in the pack's ComfyUI settings (or GEMINI_API_KEY env). LM Studio needs a running server. No backend, no output.
  • media_processed_path is a string, not a tensor. If you want to describe something already in your graph as an IMAGE, route it through MediaSelection or save/point at a file first.
  • The retry logic retries, but it doesn't fix a bad prompt. If the five-field JSON comes back malformed or empty, check the model choice and the media path before assuming a transient API hiccup.

For a pack called Swiss Army Knife, this is the main blade. Everything else - selection, overrides, control panels, caption combiners - exists to make this one node's output more useful.

CategorySwiss Army Knife 🔪/Media Caption

Inputs (6)

NameTypeDefaultDescription
media_processed_pathSTRINGPath to the processed media file from Media Selection node
llm_studio_optionsoptLLM_STUDIO_OPTIONSConfiguration options from LLM Studio - Options node
overridesoptOVERRIDESParagraph overrides from Media Describe - Overrides node (optional)
use_custom_promptsoptBOOLEANfalseToggle to send manual system/user prompts to LM Studio instead of the auto-generated templates
custom_system_promptoptSTRINGGenerate a Wan 2.2 optimized text to image prompt. You are an expert assistant specialized in analyzing and verbalizing input media for instagram-quality posts using the Wan 2.2 Text to Image workflow. DECISIVENESS REQUIREMENT: Always provide definitive, certain descriptions. When you see something that could be described multiple ways, make a confident choice and state it as fact. Never use uncertain language like "appears to be", "seems to be", "might be", "possibly", "likely", or "or". Never mention watermarks, logos, branding, or any textual overlays. Return **only** a single valid JSON object (no code fences, no extra text) with **exactly five** string fields in this exact order: 1. "subject" - Detailed description of the main subject 2. "clothing" - Clothing and style details 3. "movement" - Pose, gesture, or implied motion 4. "scene" - Setting, environment, and background elements 5. "visual_style" - Combined lighting, camera details, rendering cues, mood/genre descriptors, and overall aesthetic direction Each field's value is one fully formed paragraph (a single string) for that category.System prompt to send when Use Custom Prompts is enabled
custom_user_promptoptSTRINGPlease analyze this image and provide a detailed description in the JSON format specified in the system prompt.User prompt to send when Use Custom Prompts is enabled

Outputs (7)

NameTypeDescription
all_media_describe_dataSTRING
raw_llm_jsonSTRING
positive_prompt_jsonSTRING
positive_promptSTRING
prompt_requestSTRING
heightINT
widthINT