ComfyUI Node
SceneGen - MusicVideo
A ComfyUI node in SceneGen/MusicVideo with 42 inputs and 17 outputs.
SceneGen - MusicVideo
- audio
- reference_images
- Environment Images
- Asset Images
- Actor Images
- Scene Start Frames
- Analysis (S1)
- Style (S2)
- Palette (S3)
- Assets (S4)
- Montage (S5)
- Prompts (S6)
- Start Frames Info (S7)
- Motion Refinement (S8)
- Timeline Data (S9)
- Generation Status (S10)
- Stitching Info (S11)
- Cost Data (JSON)
- Final Video Path
◄gemini_api_keyYour_Api_key_Here►
◄replicate_api_tokenYour_Api_key_Here►
◄prompt_instructionDescribe a scene matching the music.►
◄filename_prefixscene_gen►
◄fps24.0►
◄render_modeFull Render►
◄model_textgemini-3-pro-preview►
◄model_imagegemini-3-pro-image-preview►
◄creativity0.8►
◄dynamicity0.7►
◄video_qualityMedium►
◄aspect_ratio16:9►
◄resolution_multiplier1.0►
◄enable_prompt_expansiontrue►
◄save_segmentstrue►
◄save_imagestrue►
◄save_assetstrue►
◄gemini_concurrency10►
◄replicate_concurrency15►
◄use_wan_fastfalse►
◄use_wan_2_5false►
◄use_kling_turbofalse►
◄use_omni_humanfalse►
◄use_hailuofalse►
◄use_hailuo_fasttrue►
◄use_veo_3_1false►
◄use_veo_3_1_fastfalse►
◄aggressive_edittrue►
◄word_influence0.7►
◄save_edltrue►
◄open_coffee_linktrue►
◄dialogues_genfalse►
◄open_reporttrue►
◄mix_native_audiofalse►
◄audio_volume1.0►
◄video_volume1.0►
◄normalize_audiofalse►
◄use_raw_referencesfalse►
◄verification_iterations0►
◄api_timeout280►
CategorySceneGen/MusicVideo
Inputs (42)
| Name | Type | Default | Description |
|---|---|---|---|
| audio | AUDIO | The input audio file (WAV/MP3) to analyze and generate video for. | |
| gemini_api_key | STRING | Your_Api_key_Here | Your Google Gemini API Key. Required for analysis and prompt generation. |
| replicate_api_token | STRING | Your_Api_key_Here | Your Replicate API Token. Required for video generation models. |
| prompt_instruction | STRING | Describe a scene matching the music. | Main instruction for the AI. Describe the desired mood, style, story, or specific visual elements. |
| filename_prefix | STRING | scene_gen | Prefix for all generated files (video, images, logs). |
| fps | FLOAT | 24.01–120 | Frame rate of the final output video. |
| render_mode | COMBO | Full Render | Full Render: Generates video using Replicate. Prompt Mode: Generates prompts and assets, then creates a slideshow from start frames (skips Replicate). |
| model_text | STRING | gemini-3-pro-preview | Gemini model used for text analysis, scripting, and prompting. |
| model_image | STRING | gemini-3-pro-image-preview | Gemini model used for generating start frames and assets. |
| creativity | FLOAT | 0.80–1 | 0.0 = Strict adherence to prompt. 1.0 = High hallucination/creative freedom. |
| dynamicity | FLOAT | 0.70–1 | 0.0 = Slow, static, contemplative shots. 1.0 = Fast cuts, high movement, intense action. |
| video_quality | COMBO | Medium | Controls resolution and quality settings for video models. High = 1080p (where available), Medium = 720p/768p, Low = 480p. |
| aspect_ratio | COMBO | 16:9 | Aspect ratio of the generated video. |
| resolution_multiplier | FLOAT | 1.00.5–2 | Scales the resolution of generated start frames. |
| enable_prompt_expansion | BOOLEAN | true | If True, the AI will expand your simple instructions into highly detailed visual prompts. |
| save_segments | BOOLEAN | true | If True, saves every individual video clip generated by Replicate to the output folder. |
| save_images | BOOLEAN | true | If True, saves the start frame images generated by Gemini. |
| save_assets | BOOLEAN | true | If True, saves the generated asset images (characters, props, environments). |
| gemini_concurrency | INT | 101–50 | Max parallel requests to Gemini API. |
| replicate_concurrency | INT | 151–50 | Max parallel video generation jobs on Replicate. |
| use_wan_fast | BOOLEAN | false | Enable Wan 2.5 Fast model. |
| use_wan_2_5 | BOOLEAN | false | Enable Wan 2.5 Standard model. |
| use_kling_turbo | BOOLEAN | false | Enable Kling v2.5 Turbo model. |
| use_omni_human | BOOLEAN | false | Enable OmniHuman model (good for realistic human movement). |
| use_hailuo | BOOLEAN | false | Enable Hailuo 2.3 model. |
| use_hailuo_fast | BOOLEAN | true | Enable Hailuo 2.3 Fast model. |
| use_veo_3_1 | BOOLEAN | false | Enable Google Veo 3.1 model. |
| use_veo_3_1_fast | BOOLEAN | false | Enable Google Veo 3.1 Fast model. |
| aggressive_edit | BOOLEAN | true | If True, forces fast-paced editing with cuts strictly on beat. Generates full clips but trims them aggressively. |
| word_influence | FLOAT | 0.7-1–1 | 1.0 = Literal visualization of lyrics. -1.0 = Ignore lyrics, focus on vibe/atmosphere. 0.0 = Balanced. |
| save_edl | BOOLEAN | true | If True, exports a CMX 3600 .edl file for importing the timeline into Premiere Pro/DaVinci Resolve. |
| open_coffee_link | BOOLEAN | true | Support the creator! Opens Buy Me a Coffee page after generation. |
| dialogues_gen | BOOLEAN | false | Enable dialogue generation for supported models (Veo3, Wan 2.5). Prevents trimming of dialogue shots. |
| open_report | BOOLEAN | true | Opens a live HTML report that updates in real-time during generation. |
| mix_native_audio | BOOLEAN | false | If True, mixes the audio generated by the video models (if any) with the input audio. |
| audio_volume | FLOAT | 1.00–2 | Volume level for the user input audio (1.0 = 100%). |
| video_volume | FLOAT | 1.00–2 | Volume level for the generated video audio (1.0 = 100%). |
| normalize_audio | BOOLEAN | false | If True, applies loudness normalization to the final mixed audio. |
| use_raw_references | BOOLEAN | false | If True, only generates assets that are NOT present in the provided reference images. |
| verification_iterations | INT | 00–3 | Number of verification passes to refine start frames against references. 0 = disabled, 1-3 = iterative improvement. |
| api_timeout | INT | 28060–600 | Timeout in seconds for all Gemini API calls across all stages. Default: 280 seconds (4.5 minutes). |
| reference_imagesopt | IMAGE | Optional images to use as references for style, characters, or environments. |
Outputs (17)
| Name | Type | Description |
|---|---|---|
| Environment Images | IMAGE | — |
| Asset Images | IMAGE | — |
| Actor Images | IMAGE | — |
| Scene Start Frames | IMAGE | — |
| Analysis (S1) | STRING | — |
| Style (S2) | STRING | — |
| Palette (S3) | STRING | — |
| Assets (S4) | STRING | — |
| Montage (S5) | STRING | — |
| Prompts (S6) | STRING | — |
| Start Frames Info (S7) | STRING | — |
| Motion Refinement (S8) | STRING | — |
| Timeline Data (S9) | STRING | — |
| Generation Status (S10) | STRING | — |
| Stitching Info (S11) | STRING | — |
| Cost Data (JSON) | STRING | — |
| Final Video Path | STRING | — |