Nodes/VRGameDevGirl Video Enhancement Nodes/๐ŸŒ€ VRGDG_MusicVideoPromptCreatorJson
ComfyUI Node

๐ŸŒ€ VRGDG_MusicVideoPromptCreatorJson

The Brain of the Music-Video Workflow (Yes, It Needs a Gemini API Key)

By vrgamegirl19ยทCreated about a year agoยทUpdated about 7 hours agoยท 718
๐ŸŒ€ VRGDG_MusicVideoPromptCreatorJson
  • signal
  • concatenated_string
โ—„character_descriptionThe Woman.โ–บ
โ—„song_theme_stylecinematic realism, emotional storytelling, soft surrealism, naturalistic tone, dreamlike nostalgia, modern drama, poetic symbolism, intimate atmosphereโ–บ
โ—„pipe_separated_lyricsline1 | line2 | line3โ–บ
โ—„word_count_min30โ–บ
โ—„word_count_max50โ–บ
โ—„list_handling_modeReference Guide (LLM creates variations inspired by list)โ–บ
โ—„prompt_structure_modeCharacter-Focused (character always leads)โ–บ
โ—„environmentopen field at dusk, dimly lit bedroom, empty city street at night, forest clearing with morning fog, seaside cliff at golden hour, rainy urban alley, sunlit living room, desert road at sunriseโ–บ
โ—„lightingwarm amber glow, cool window light, neon reflections, diffused morning light, soft backlight haze, flickering streetlights, gentle afternoon sun, pink-orange dawn lightโ–บ
โ—„camera_motionzoom in, zoom out, tilt down, rotate around, tilt up, pan, trackโ–บ
โ—„physical_interactionwalking through tall grass, lying on bed staring upward, leaning against a wall in stillness, reaching toward sunlight, hair moving in wind, footsteps in puddles, brushing hand across furniture, standing motionless in breezeโ–บ
โ—„facial_expressionIntense raw emotionโ–บ
โ—„shotsClose up, medium, wide angle, over the shoulder, point of view, overhead, ground levelโ–บ
โ—„outfit_rulesa white dressโ–บ
โ—„character_visibilitymostly visible, half-shadowed, silhouetted, reflected or obscured, seen from behind, partially out of frame, emerging from light, fading into darknessโ–บ
โ—„story_modefalseโ–บ
โ—„custom_instructionsโ–บ
โ—„Summary_File_Pathโ–บ
โ—„summary_index0โ–บ
โ—„total_sets1โ–บ
โ—„groups_in_last_set16โ–บ

VRGDG_MusicVideoPromptCreatorV3 is the node that turns lyrics, a character, and a pile of style lists into a full set of scene prompts for a music video. It's an LLM call with a fancy front end - and that means it needs a Gemini API key, which is the first thing people miss about it.

Let's be upfront about the architecture, because it changes everything: this node does not generate prompts locally. It sends your inputs to Google's Gemini API and returns the assembled scene prompts as the single concatenated_string output. The name and the "V3" and the "Json" in the display name hint at a long iteration history; what you get today is the orchestration hub of the whole pack. No key, no output - it just errors. Get one from the Google AI Studio / Gemini API console; it's a paid API, though cheap for text.

Inputs that matter

The defaults are thoughtfully pre-filled, which is rare and good. You'll change a handful:

  • character_description - who the video stars. Default "The Woman." Keep it tight; this gets repeated into every scene prompt.
  • song_theme_style - the aesthetic envelope: mood words, genre feel, color language.
  • pipe_separated_lyrics - your lyrics as line1 | line2 | line3. This is where cleaned lyric output from the extractor/cleaner nodes lands, and it's what makes the video actually follow the song.
  • word_count_min / word_count_max - how long each scene prompt should be (defaults 30โ€“50 words). LTX-style video models read best at these lengths; don't crank them up thinking longer is better.
  • list_handling_mode - how the lists below get used: Strict Cycle (each item used once, then repeat), Reference Guide (LLM writes variations inspired by the list - the default), Random Selection, or Free Interpretation.
  • prompt_structure_mode - the sentence skeleton: character-first, environment-first, action-first, rotating, or LLM's choice.

Then come the content lists, all multiline: environment, lighting, camera_motion, physical_interaction, facial_expression, shots, outfit_rules, character_visibility. Each ships with a solid default pool, and these are the dials that actually shape the look - swap the environment list and the whole video relocates.

The story mode

Toggle story_mode and the node tracks continuity across runs using summary files. The optional Summary_File_Path + summary_index load a previous run's summary (so a multi-part video stays consistent), and total_sets / groups_in_last_set tell the LLM which run is last and how many prompts to leave room for. The tooltip is blunt: story mode is advanced and "using Gemini PRO is recommended" - meaning expect heavier API usage and better results on the bigger model. It's the difference between "sixteen disconnected scenes" and "a sixteen-scene video that remembers what happened in scene one."

Where it sits in the pack

This is the node everything else feeds: lyrics upstream, character and style here, and the concatenated_string output flows into the chunker (VRGDG_IndexedPromptChunker) which splits it into per-scene prompts for generation. The whole music-video pipeline exists to serve this one call and distribute its result.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/vrgamegirl19/comfyui-vrgamedevgirl

Restart ComfyUI; it's under VRGDG/Prompt Tools. ComfyUI Manager: search vrgamedev. No extra local models - the dependency is the API key, not a download.

Honest notes

Three things. First, you're at Gemini's mercy for censorship and tone - Google's image models are famously filtered, and while this is text-only, the prompt text you get back still inherits the model's sensibilities; the KB's coverage of closed-source models calls Gemini "heavily censored" and "unimaginative" for image generation, and the same guardrails color its text output. Second, it costs money per run, and story mode multiplies that. Third, the README doesn't document this node at all - trust the tooltips, and test one song's worth of prompts before committing to a long run. Despite the caveats, it's the most genuinely useful node in the pack: the defaults are good, and the result is a shot list you'd otherwise spend an hour hand-writing.

CategoryVRGDG/Prompt Tools

Inputs (22)

NameTypeDefaultDescription
character_descriptionSTRINGThe Woman.โ€”
song_theme_styleSTRINGcinematic realism, emotional storytelling, soft surrealism, naturalistic tone, dreamlike nostalgia, modern drama, poetic symbolism, intimate atmosphereโ€”
pipe_separated_lyricsSTRINGline1 | line2 | line3โ€”
word_count_minINT3010โ€“200โ€”
word_count_maxINT5010โ€“200โ€”
list_handling_modeCOMBOReference Guide (LLM creates variations inspired by list)4 options: Strict Cycle (use each once, then repeat), Reference Guide (LLM creates variations inspired by list), Random Selection (pick randomly from list), Free Interpretation (LLM can ignore or combine items)
prompt_structure_modeCOMBOCharacter-Focused (character always leads)5 options: Character-Focused (character always leads), Environment-Focused (setting always leads), Action-Focused (movement always leads), Cycle Through Patterns (rotate 3 structures), Dynamic Choice (LLM picks best structure per lyric)
environmentSTRINGopen field at dusk, dimly lit bedroom, empty city street at night, forest clearing with morning fog, seaside cliff at golden hour, rainy urban alley, sunlit living room, desert road at sunriseโ€”
lightingSTRINGwarm amber glow, cool window light, neon reflections, diffused morning light, soft backlight haze, flickering streetlights, gentle afternoon sun, pink-orange dawn lightโ€”
camera_motionSTRINGzoom in, zoom out, tilt down, rotate around, tilt up, pan, trackโ€”
physical_interactionSTRINGwalking through tall grass, lying on bed staring upward, leaning against a wall in stillness, reaching toward sunlight, hair moving in wind, footsteps in puddles, brushing hand across furniture, standing motionless in breezeโ€”
facial_expressionSTRINGIntense raw emotionโ€”
shotsSTRINGClose up, medium, wide angle, over the shoulder, point of view, overhead, ground levelโ€”
outfit_rulesSTRINGa white dressโ€”
character_visibilitySTRINGmostly visible, half-shadowed, silhouetted, reflected or obscured, seen from behind, partially out of frame, emerging from light, fading into darknessโ€”
story_modeBOOLEANfalseAdvanced โ€” using Gemini PRO is recommended
signal*โ€”
custom_instructionsoptSTRINGOptional: Enter your own custom prompt instructions. If filled, this will override the normal system instructions.
Summary_File_PathoptSTRINGIf provided, this summary will override the theme/style in story mode.
summary_indexoptINT00โ€“999Which summary file to load (e.g. 2 loads summary2.json)
total_setsoptINT11โ€“100Used to determine if the current run is the last one (which does not require summary output).
groups_in_last_setoptINT161โ€“16Used in story mode to tell the LLM how many prompts to generate in the final run (e.g., if fewer than 16).

Outputs (1)

NameTypeDescription
concatenated_stringSTRINGโ€”