SceneGen - MusicVideo

This node does the thing you've daydreamed about while scrubbing Wan clips: drop a song in, get a finished, beat-synced music video out the other end. Not a workflow you stitch together - a whole production pipeline in one node, from lyric analysis to the final .mp4. It's the most ambitious API-wrapper node I've seen in ComfyUI, and the most expensive one you'll ever queue.

Scene Gen is the work of an independent dev, Paul Lazniak, and it's a small, obscure pack with almost no community chatter - read this before you hit queue. Everything below applies to its twin, SceneGen - Movies, too.

What it actually is

SceneGenMusicVideoNode is a pure cloud pipeline in a node. Google's Gemini 3 Pro does the "thinking" - it listens to your audio, writes the visual plan, designs assets, and generates start frames. Replicate runs the actual video models: Wan 2.5, Kling v2.5 Turbo, Hailuo 2.3, Veo 3.1, and OmniHuman. You need two API keys, one for each service. No models download to your machine.

The framing from the external-api-nodes playbook: the "model isn't on your machine" move, every one of these video models a closed, pay-per-call service. The cost is real, and it's what people underestimate most.

How the 11-stage pipeline works

When you queue it, the node walks stages S1–S11, watchable live in the HTML report (open_report, on by default). The shape:

• S1–S5 (analysis): the raw audio goes to Gemini as WAV bytes, which analyzes genre, mood, lyrics, and structure, then produces a style, a color palette, and a montage cut to the song's timing.
• S4 (assets): Gemini's image model generates consistent environments, props, and character sheets before "filming" starts.
• S7 (start frames): each shot gets an AI-generated first frame to animate from.
• S9–S11 (production): every shot goes to Replicate, comes back, gets trimmed to the timeline, and is stitched and muxed with your original audio. Optionally you get a CMX 3600 .edl for Premiere or DaVinci Resolve.

That EDL export is the sleeper feature - rare in a ComfyUI node, and it means the result isn't a dead end.

The inputs that matter

Most of the 40+ inputs have sane defaults. These change the result:

• audio - connect a standard Load Audio node. WAV or MP3 both work.
• gemini_api_key / replicate_api_token - paste your keys here. The placeholder literally reads Your_Api_key_Here; if you leave it, the run dies on the first auth call.
• prompt_instruction - your one-line art direction ("a neon cyberpunk chase", "a calm nature documentary"). With enable_prompt_expansion on (default), Gemini turns it into full shot prompts.
• creativity, dynamicity, word_influence - the three creative sliders. word_influence is the fun one: 1.0 visualizes lyrics literally ("fire" shows fire), −1.0 ignores words for pure vibe.
• render_mode - the cost control. Full Render spends Replicate money. Prompt Mode runs to the start frames, skips Replicate, and makes a slideshow instead - your free way to test a track.
• The use_* model toggles - here's a trap: only use_hailuo_fast is on by default. Wan, Kling, Veo, OmniHuman are all off until you flip them. Want Veo 3.1? Enable it yourself.

Outputs

The four image outputs - Environment Images, Asset Images, Actor Images, Scene Start Frames - are the art direction as previewable tensors. The S1–S11 strings are the audit trail, JSON explaining every decision. The two that matter downstream: Timeline Data (S9) feeds the pack's Scene Gen Data Extractor (the free local-render trick), and Final Video Path points at your finished .mp4. Watch Cost Data (JSON), too.

Install

ComfyUI Manager → search "Scene Gen", or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/lazniak/scene_gen.git
cd scene_gen
pip install -r requirements.txt

Then restart ComfyUI. Dependencies are light - google-generativeai, numpy, scipy, replicate, requests - and there are no model files to fetch. Get the Gemini key from Google AI Studio and the Replicate token from your account settings.

Where people get burned

• The bill. Every shot is a separate Replicate job, and aggressive_edit (on by default) may generate extra footage to hit the beats - a 3-minute song can mean dozens of paid clips. Veo and Kling are the priciest. Test in Prompt Mode first.
• The coffee tab. open_coffee_link (on by default) opens the author's Buy Me a Coffee page after every run. Turn it off.
• Timeouts. api_timeout defaults to 280 seconds; long analyses can exceed it on slow connections. Bump it before blaming the node.
• It's a phone-home node. It holds your keys and sends your audio and prompts to Google and Replicate by design - same posture as every API wrapper, and this category has shipped malware before. Install from the official repo.
• License. It's custom and non-commercial, so don't sell the output.