Nodes/Scene Gen - Agent for music videos/SceneGen - Movies (In Development)
ComfyUI Node

SceneGen - Movies (In Development)

The same engine, an in-development label, and a dialogue toggle

By lazniak·Created 10 months ago·Updated 10 months ago· 4
SceneGen - Movies (In Development)
  • audio
  • reference_images
  • Environment Images
  • Asset Images
  • Actor Images
  • Scene Start Frames
  • Analysis (S1)
  • Style (S2)
  • Palette (S3)
  • Assets (S4)
  • Montage (S5)
  • Prompts (S6)
  • Start Frames Info (S7)
  • Motion Refinement (S8)
  • Timeline Data (S9)
  • Generation Status (S10)
  • Stitching Info (S11)
  • Cost Data (JSON)
  • Final Video Path
gemini_api_keyYour_Api_key_Here
replicate_api_tokenYour_Api_key_Here
prompt_instructionDescribe a scene matching the music.
filename_prefixscene_gen
fps24.0
render_modeFull Render
model_textgemini-3-pro-preview
model_imagegemini-3-pro-image-preview
creativity0.8
dynamicity0.7
video_qualityMedium
aspect_ratio16:9
resolution_multiplier1.0
enable_prompt_expansiontrue
save_segmentstrue
save_imagestrue
save_assetstrue
gemini_concurrency10
replicate_concurrency15
use_wan_fastfalse
use_wan_2_5false
use_kling_turbofalse
use_omni_humanfalse
use_hailuofalse
use_hailuo_fasttrue
use_veo_3_1false
use_veo_3_1_fastfalse
aggressive_edittrue
word_influence0.7
save_edltrue
open_coffee_linktrue
dialogues_genfalse
open_reporttrue
mix_native_audiofalse
audio_volume1.0
video_volume1.0
normalize_audiofalse
use_raw_referencesfalse
verification_iterations0
api_timeout280

If you've read the SceneGen - MusicVideo article, you already know this node. SceneGen - Movies (In Development) is the same engine with a different menu label: open the right-click menu and you'll find it under SceneGen → Movies → SceneGen - Movies, while its twin lives under SceneGen → MusicVideo. Look at the inputs side by side and you'll find they're identical, down to the last slider.

So why two nodes? The author's framing in the README is that MusicVideo is for syncing visuals to beats and lyrics, while Movies targets "narrative storytelling, dialogue, and script-based content." The honest read: the label is the roadmap, and the code is the same 11-stage Gemini + Replicate pipeline you get with the music video node. The one meaningful difference is that the Movies variant is explicitly marked "In Development," which is your cue not to expect the movie workflow to feel finished yet.

What's actually different

Not the inputs. You'll find the full set here - audio, gemini_api_key, replicate_api_token, prompt_instruction, render_mode, creativity, dynamicity, word_influence, the use_* video model toggles, and so on. The outputs match too: the four image tensors (Environment Images, Asset Images, Actor Images, Scene Start Frames), the S1–S11 JSON audit trail, Cost Data (JSON), and Final Video Path.

The only field that hints at the narrative ambition is dialogues_gen. When enabled, the node asks Gemini to write dialogue for supported models (Veo 3.1 and Wan 2.5) and prevents those dialogue shots from being trimmed on beat, which would wreck the pacing of a scene. That's the one real "movies" behavior in the pack, and it's on by default in neither node - you have to flip it.

Everything else - the two API keys, the cost math, the render_mode trick for free testing, the model toggles that all default to off except Hailuo Fast - is identical to the MusicVideo node, so I won't repeat it here. If you're about to queue this for the first time, read that article's install and troubleshooting sections; they apply word for word.

My take

As of today, this node is a placeholder for a promise. The pipeline genuinely works - it's the same one that produces finished, beat-synced music videos - but "movies" as a distinct capability (real dialogue, scene-level continuity, narrative structure that survives an aggressive edit) isn't there yet. If you want to make a music video, use it or its twin; if you want to make a movie, this won't do it for you yet. Don't let the label sell you on a finished feature.

Worth saying: this is an obscure, single-dev pack with almost no community footprint, so the maintenance risk is real. If you build a workflow on dialogues_gen, check the repo for updates before assuming a bug is yours.

Install

Same as the pack's other nodes. ComfyUI Manager → search "Scene Gen", or:

cd ComfyUI/custom_nodes
git clone https://github.com/lazniak/scene_gen.git
cd scene_gen
pip install -r requirements.txt

Restart ComfyUI, then paste your Google Gemini key and Replicate token into the node fields. No model downloads - all inference happens in the cloud.

CategorySceneGen/Movies (In Development)

Inputs (42)

NameTypeDefaultDescription
audioAUDIOThe input audio file (WAV/MP3) to analyze and generate video for.
gemini_api_keySTRINGYour_Api_key_HereYour Google Gemini API Key. Required for analysis and prompt generation.
replicate_api_tokenSTRINGYour_Api_key_HereYour Replicate API Token. Required for video generation models.
prompt_instructionSTRINGDescribe a scene matching the music.Main instruction for the AI. Describe the desired mood, style, story, or specific visual elements.
filename_prefixSTRINGscene_genPrefix for all generated files (video, images, logs).
fpsFLOAT24.01–120Frame rate of the final output video.
render_modeCOMBOFull RenderFull Render: Generates video using Replicate. Prompt Mode: Generates prompts and assets, then creates a slideshow from start frames (skips Replicate).
model_textSTRINGgemini-3-pro-previewGemini model used for text analysis, scripting, and prompting.
model_imageSTRINGgemini-3-pro-image-previewGemini model used for generating start frames and assets.
creativityFLOAT0.80–10.0 = Strict adherence to prompt. 1.0 = High hallucination/creative freedom.
dynamicityFLOAT0.70–10.0 = Slow, static, contemplative shots. 1.0 = Fast cuts, high movement, intense action.
video_qualityCOMBOMediumControls resolution and quality settings for video models. High = 1080p (where available), Medium = 720p/768p, Low = 480p.
aspect_ratioCOMBO16:9Aspect ratio of the generated video.
resolution_multiplierFLOAT1.00.5–2Scales the resolution of generated start frames.
enable_prompt_expansionBOOLEANtrueIf True, the AI will expand your simple instructions into highly detailed visual prompts.
save_segmentsBOOLEANtrueIf True, saves every individual video clip generated by Replicate to the output folder.
save_imagesBOOLEANtrueIf True, saves the start frame images generated by Gemini.
save_assetsBOOLEANtrueIf True, saves the generated asset images (characters, props, environments).
gemini_concurrencyINT101–50Max parallel requests to Gemini API.
replicate_concurrencyINT151–50Max parallel video generation jobs on Replicate.
use_wan_fastBOOLEANfalseEnable Wan 2.5 Fast model.
use_wan_2_5BOOLEANfalseEnable Wan 2.5 Standard model.
use_kling_turboBOOLEANfalseEnable Kling v2.5 Turbo model.
use_omni_humanBOOLEANfalseEnable OmniHuman model (good for realistic human movement).
use_hailuoBOOLEANfalseEnable Hailuo 2.3 model.
use_hailuo_fastBOOLEANtrueEnable Hailuo 2.3 Fast model.
use_veo_3_1BOOLEANfalseEnable Google Veo 3.1 model.
use_veo_3_1_fastBOOLEANfalseEnable Google Veo 3.1 Fast model.
aggressive_editBOOLEANtrueIf True, forces fast-paced editing with cuts strictly on beat. Generates full clips but trims them aggressively.
word_influenceFLOAT0.7-1–11.0 = Literal visualization of lyrics. -1.0 = Ignore lyrics, focus on vibe/atmosphere. 0.0 = Balanced.
save_edlBOOLEANtrueIf True, exports a CMX 3600 .edl file for importing the timeline into Premiere Pro/DaVinci Resolve.
open_coffee_linkBOOLEANtrueSupport the creator! Opens Buy Me a Coffee page after generation.
dialogues_genBOOLEANfalseEnable dialogue generation for supported models (Veo3, Wan 2.5). Prevents trimming of dialogue shots.
open_reportBOOLEANtrueOpens a live HTML report that updates in real-time during generation.
mix_native_audioBOOLEANfalseIf True, mixes the audio generated by the video models (if any) with the input audio.
audio_volumeFLOAT1.00–2Volume level for the user input audio (1.0 = 100%).
video_volumeFLOAT1.00–2Volume level for the generated video audio (1.0 = 100%).
normalize_audioBOOLEANfalseIf True, applies loudness normalization to the final mixed audio.
use_raw_referencesBOOLEANfalseIf True, only generates assets that are NOT present in the provided reference images.
verification_iterationsINT00–3Number of verification passes to refine start frames against references. 0 = disabled, 1-3 = iterative improvement.
api_timeoutINT28060–600Timeout in seconds for all Gemini API calls across all stages. Default: 280 seconds (4.5 minutes).
reference_imagesoptIMAGEOptional images to use as references for style, characters, or environments.

Outputs (17)

NameTypeDescription
Environment ImagesIMAGE
Asset ImagesIMAGE
Actor ImagesIMAGE
Scene Start FramesIMAGE
Analysis (S1)STRING
Style (S2)STRING
Palette (S3)STRING
Assets (S4)STRING
Montage (S5)STRING
Prompts (S6)STRING
Start Frames Info (S7)STRING
Motion Refinement (S8)STRING
Timeline Data (S9)STRING
Generation Status (S10)STRING
Stitching Info (S11)STRING
Cost Data (JSON)STRING
Final Video PathSTRING