Generate long-form AI video with lipsync, custom audio, background ambience, per-segment prompts, and consistent character identity across a whole clip — all from a single node with a built-in visual timeline editor.

What it does

A full timeline-based node for generating long-form AI video with LTX 2.3 AV — chunked generation, reference-frame continuity, per-segment prompts, lipsync, layered audio, and character-reference guidance — from a single node.

Key features

Visual timeline editor — drag-and-drop images, audio, and video segments directly onto the timeline
Character-reference guide — up to three character "slots" in the timeline's own character-card panel (drop image, optional VLM Analyze), locked into the generation via one of two mechanisms:
- Ghost Mask (End) — reference images are appended as hidden guide frames past the end of each chunk, then cropped off before decode, so the model sees real reference photos as identity anchors without them ever appearing in the output. If a chunk's prompt tags specific slots with @ref1/@ref2/@ref3, only those slots' references are sent for that chunk — untagged prompts fall back to sending every filled slot, same as before.
- Licon MSR (Prefix) — a real IC-LoRA identity guide (via ComfyUI-LTXVideo's LTXAddVideoICLoRAGuide/LTXVAddGuide) injected as a prefix rather than a hidden tail. Needs vae connected. Already respects @refN tags the same way.
Image Anchors — drop any image or video onto the timeline and it gets its own prompt box; right-click to convert it to a guide-only "anchor" that borrows the surrounding prompt zone instead of owning its own.
Seed Hunt — cheap 4-seed scouting pass at Stage 1 resolution before committing to a full render; picking a candidate carries its actual cached Stage 1 latent into Stage 2, not a fresh independent roll.
Infinite chunking — generate 90 seconds+ by splitting into overlapping chunks with carry-frame latent locking for seamless transitions, with automatic chunk-boundary alignment to timeline segments (auto_chunk_by_segment).
Per-segment prompting — different prompts for different parts of the video using PromptRelay temporal attention masking, plus segment_override_1..8 sockets for wiring in LLM-generated prompts.
Lipsync — sync mouth movements to a custom speech audio file using the LTX talking head LoRA.
Three audio layers — generated ambient audio, custom speech/music, and a separate BG ambience track.
IC-LoRA motion support — reference video for camera motion or style consistency.
Retake mode — replace a section of an existing video with a new generation.
Dual-pass sampling — Stage 1 draft + Stage 2 spatial-upscale refine (or single_stage_mode to skip straight to one full-resolution pass).
Real NAG negative-prompt support — via ComfyUI-KJNodes' LTX2_NAG.
Color matching — automatic color correction between chunks.

Nodes included

| Node | Description | |------|-------------| | MuseDirectorSamplerV10Final | Main director + infinite sampler, with timeline UI and character-reference guide | | MuseGuide | Reference keyframe, IC-LoRA guide, and Licon MSR identity encoder — wires directly into LTX conditioning | | MuseCropGuides | Trims guide keyframes to match a cropped or upscaled latent — use after the spatial upscaler |

Requirements

ComfyUI custom nodes (install via Manager)

ComfyUI-VideoHelperSuite — required for video preview and saving
ComfyUI-LTXVideo — required only for Licon MSR (Prefix) reference mode
ComfyUI-KJNodes — required for NAG negative-prompt support

Python packages

pip install av torchaudio soundfile

Installation

cd ComfyUI/custom_nodes
git clone https://github.com/muse-collective-26/muse-director-v10-final

Restart ComfyUI after installing.

Model setup

Download all models and place them in the correct subfolders inside your ComfyUI models/ directory.

Text Encoder (CLIP)

Download: gemma_3_12B_it_fp4_mixed.safetensors Place in: models/text_encoders/

Spatial Upscaler

Download: ltx-2.3-spatial-upscaler-x2-1.1.safetensors Place in: models/latent_upscale_models/

Talking Head LoRA (required for Lipsync)

Download: LTX-2.3-22b-AV-LoRA-talking-head-v1.safetensors Place in: models/loras/LTX2.3/

Licon MSR identity LoRA (required only for Licon MSR (Prefix) reference mode)

Load via a LoraLoaderModelOnly node upstream of this node's model input — not via any ic_lora_name widget, which is a separate, motion-guidance-only feature.

Node connections

| Input | Type | Description | |-------|------|--------------| | model | MODEL | LTX model with LoRA applied (use talking head LoRA if using lipsync; identity LoRA if using Licon MSR) | | clip | CLIP | LTX text encoder | | audio_vae | VAE | LTX Audio VAE | | vae | VAE | LTX Video VAE — required for Licon MSR (Prefix) | | spatial_upscaler | LATENT_UPSCALE_MODEL | LTX spatial upscaler | | ref_images | IMAGE (optional) | Extra non-character reference material (e.g. an object) — single image or batch, appended after the timeline's character-card images | | bg_audio | AUDIO (optional) | Background ambience — mixed under everything | | base_model | MODEL (optional) | Base model without LoRA — wire the UNETLoader output directly here for ambient audio generation without the talking head LoRA influencing it |

Full settings reference

This node has a lot of widgets — most people only ever touch a handful of them (the ones covered above and in the timeline UI itself). This table covers every single one, including the deeper sampling/guide controls that aren't visible unless you actually go looking, so nothing on the node is a mystery.

Generation & Resolution

| Setting | Description | |---|---| | frame_rate | Output frame rate, default 24 | | custom_width / custom_height | Target resolution (default 960×544) | | resize_method | How reference/guide images that don't match the target resolution get fit: maintain aspect ratio (default, letterboxes), stretch to fit (distorts), crop, pad, pad green (pads with a chroma-key green for later keying out) | | divisible_by | Resolution rounding constraint (default 32 — LTX's own requirement) | | img_compression | JPEG-style compression applied to reference images before guide encoding (0–51, default 18) — higher softens fine detail on the reference itself | | filename_prefix | Output filename prefix | | bg_volume | Background-audio track volume multiplier |

Audio

| Setting | Description | |---|---| | generate_audio | Ambient/SFX audio from [SOUNDS] prompt tags | | custom_audio_on | Play the AUDIO timeline track's own file(s) | | lipsync | Sync mouth movement to custom audio (needs custom_audio_on + talking head LoRA) | | enable_ambient_pass | A second, LoRA-free ambient/SFX pass layered under the main speech — the talking-head LoRA otherwise suppresses ambient sound entirely, in both audio modes. Genuine tradeoff: this pass watches the actual video as visual context to generate matching sound, which can occasionally produce a faint duplicate/echoed voice. Default ON; try OFF if you're chasing that specific artifact. | | bg_audio (input) | Background ambience track, mixed under everything at bg_volume |

Chunking

| Setting | Description | |---|---| | chunk_duration_seconds | Length of each generated chunk (default 10s) | | auto_chunk_threshold | Videos longer than this get split into chunks automatically | | auto_chunk_by_segment | ON: chunk boundaries exactly match your timeline segment boundaries (ignores the two settings above). OFF (default): fixed-duration chunking, a segment may straddle a chunk boundary | | carry_frames | Frames locked from the previous chunk as a continuity reference (default 73 ≈ 3s at 24fps) | | carry_strength | How strongly carry frames anchor the next chunk (0–1, default 1.0) | | crossfade_frames | Blend frames at chunk boundaries to smooth the transition |

Sampling — Stage 1 / Stage 2

This node runs a two-stage pipeline by default: a cheap low-res Stage 1 draft, then a Stage 2 pass that upscales and refines it. This is invisible unless you go looking — the two stages share one set of character-reference/guide settings but have entirely separate step counts and samplers.

| Setting | Description | |---|---| | stage1_steps | Stage 1 draft-pass step count (default 8 — deliberately low, it's a draft) | | stage2_steps | Stage 2 refine-pass step count (default 4) | | stage2_denoise | How much Stage 2 reworks Stage 1's draft (0–1, default 0.42) — higher values deviate further from the draft | | cfg | Classifier-free guidance scale (default 1.0). Kept at 1.0 deliberately — real negative-prompt steering comes from NAG below instead, not the classic CFG negative branch | | single_stage_mode | ON: skip Stage 2 entirely, sample once directly at full resolution — stage1_steps becomes the only pass's step count (raise it well above the draft-only default of 8 if you use this). Seed Hunt is ignored while this is on, since there's no Stage 2 for a scouted candidate to refine into. Experimental — no confound-free quality comparison against the normal two-stage flow yet | | stage1_sampler / stage2_sampler | Sampler algorithm per stage (default euler for both) | | seed | Sampler seed |

Guide settings (advanced)

These control how the character-reference guide (Ghost Mask or Licon MSR) actually gets encoded and blended in — most people never need to touch these, but they're here if a specific reference isn't behaving.

| Setting | Description | |---|---| | guide_scale_by / guide_scale_by_s2 | Guide strength scaling applied at Stage 1 / Stage 2 respectively (default 0.5 / 1.0) | | guide_upscale_method | Interpolation method used when resizing guide images (bicubic, bilinear, nearest-exact, area, bislerp) | | guide_image_attn_strength | 0–1, how strongly the guide's cross-attention influences generation (default 1.0) | | guide_crop | center (default) or disabled — whether guide images get center-cropped to fit | | guide_auto_snap_ic_grid | ON (default): automatically snap guide dimensions to the IC-LoRA's expected grid | | guide_use_tiled_encode | ON: tile the guide-image VAE encode to save memory on large/high-res references, at some speed cost. Off by default | | guide_tile_size / guide_tile_overlap | Tile size/overlap when tiled encode is on | | ghost_anchor_buffer | Ghost Mask (End) only. Extra empty latent frames inserted between the real visible content and the hidden reference tail (default 2) — raise this if you see faint artifacting building up in the last several visible frames of a chunk near the anchor |

Reference / character guide

| Setting | Description | |---|---| | reference_mode | OFF, Ghost Mask (End), or Licon MSR (Prefix) — see Key Features above | | reference_strength | Guide strength for character/ref reference images (0–5, default 1.0) | | msr_prefix_frames | Licon MSR (Prefix) only. Pixel-frame budget for the reference slideshow, shared across however many identity images + background you've provided (default 65) — more reference images means less budget per image unless you raise this. Should be 1 + a multiple of 8; other values are floored to the nearest valid count automatically | | ref_images (input) | Extra reference image(s) beyond the character cards — a single image or batch, e.g. an object rather than a person |

NAG (negative-prompt guidance)

Real negative-prompt steering, via ComfyUI-KJNodes' LTX2_NAG — necessary because cfg is fixed at 1.0, which neutralizes the classic negative-prompt branch.

| Setting | Description | |---|---| | negative_prompt | What to steer away from (e.g. "moles, blemishes, skin spots"). Empty = no effect | | nag_scale | Strength of the negative-guidance effect (default 11.0). 0 disables it | | nag_alpha | 0–1, default 0.25 | | nag_tau | 0–10, default 2.5 | | nag_bypass | Hard kill-switch — ON means NAG is never touched at all, regardless of the other NAG settings. Checked first |

Automation & overrides

For wiring this node into a larger automated pipeline rather than driving it by hand from the timeline UI.

| Setting | Description | |---|---| | automation_start / automation_end / automation_duration (inputs) | Connection-only (no widget row) — override the timeline panel's own start/end/duration in seconds when wired in. start+end connected together derive the length; duration connected overrides length directly | | optional_latent (input) | Override the auto-generated empty latent for chunk 1 only — ignored if its shape doesn't match, or on chunk 2+ | | segment_override_1..8 (inputs) | Connection-only, 0-indexed (segment_override_1 = segment 0's prompt, etc.) — overrides that segment's prompt text when connected and non-empty. Timing/images still come from the timeline as normal. The "Segment N Prompt" boxes in the timeline UI are the only place you type text directly; these sockets exist purely for wiring in something like MusePromptSplitter's output (an LLM-expanded prompt) as an override |

Outputs

| Output | Type | Description | |---|---|---| | last_chunk_frames | IMAGE | The final chunk's decoded frames — wire into a Video Combine node for your finished clip | | audio | AUDIO | The finished mixed audio track | | stage1_frames | IMAGE | Stage 1's own draft-resolution decode, useful for debugging what the low-res pass actually produced before Stage 2 touched it | | seed_hunt_preview_1..4 | IMAGE | The four Seed Hunt scouting candidates — see Seed Hunt below | | seed_hunt_audio_1..4 | AUDIO | Each Seed Hunt candidate's own audio | | reference_image | IMAGE | The actual reference image used for the character guide this run — useful for feeding into a separate color-match node so the final output's color grade matches the reference (see Example workflow below) |

Seed Hunt, in detail

LTX 2.3 is far more sensitive to seed than to prompt wording — the fastest path to a good result is usually trying a handful of seeds cheaply and picking the best one, rather than iterating on the prompt. Seed Hunt bakes that workflow into the node itself, using the node's own real timeline data (images, audio, motion guide) so multi-segment timelines just work with no extra wiring.

How it works:

Flip seed_hunt ON, leave all four use_seed_hunt_1..4 toggles OFF, and run. Instead of the full Stage 1 + Stage 2 pipeline, the node generates 4 candidates at Stage 1's real resolution, each with a fresh random seed. The seed_hunt_preview_1..4 / seed_hunt_audio_1..4 outputs show each one, and each candidate's actual Stage 1 latent is cached in memory.
Look at the 4 previews (wire each to its own Video Combine node so you can actually watch them), then flip exactly one use_seed_hunt_N ON and run again. The node skips regenerating Stage 1 from scratch and instead carries that candidate's cached real latent straight into Stage 2 (upscale + refine). What you saw in the preview is what carries through — not a fresh independent roll that merely happens to share a seed number.
If the cache is empty (e.g. the server restarted since scouting, or you commit to a candidate without ever scouting first), it falls back to overriding seed and running Stage 1 fresh, using that candidate's seed_hunt_N widget value.

seed_hunt_scale and seed_hunt_1 are legacy widgets kept only so older saved workflows still load correctly — scouting now always runs at Stage 1's real resolution with a fresh random seed per candidate, regardless of what those two are set to.

single_stage_mode and Seed Hunt are mutually exclusive — Seed Hunt is ignored while single-stage mode is on, since there's no Stage 2 for a scouted candidate to be refined into.

Example workflow

A full production workflow is included at workflows/Muse-Director-V10.json — the same one used to produce real output with this node, including Seed Hunt scouting (4 candidates, each with its own preview/duration/segment breakdown), Stage 1 preview, spatial upscaling, batched color-matching against the original reference, and an LLM-based prompt-expansion stage feeding the segment_override_1..8 sockets.

It uses several extra nodes on top of the core requirements above:

ComfyUI-KJNodes (ModelPreviewOverrideKJ) — model-preview wrapper, purely decorative
ComfyUI-LayerStyle (LayerUtility: PurgeVRAM) — clears VRAM between stages, not required for generation to work
comfyui_nvidia_rtx_nodes (RTXVideoSuperResolution) — RTX-accelerated upscaling on the "Upscaled" output branch; needs an RTX GPU
muse-collective-batched-colour-match (MuseColorMatchBatched) — a separate, memory-bounded color-match node, installed independently — takes this node's reference_image output to grade the final video's colors to match the original reference photo
muse-ltx-timeline (MusePromptSplitter) — parses the LLM's expanded PROMPT_N/DURATION_N output (see the GeminiNodeV2 node's system prompt in the workflow) and splits it out to the segment_override_1..8 sockets
A handful of PreviewAny/PreviewImage/VHS_VideoCombine nodes are all stock ComfyUI/VideoHelperSuite — nothing extra needed for those, they're just wired up multiple times for scouting/debugging visibility

GeminiNodeV2 (the LLM prompt-expansion node) ships with ComfyUI core as of recent versions — no separate install, but it does need a Gemini API key configured in ComfyUI's own settings to actually run. Its system prompt (visible if you open that node in the workflow) is a genuinely useful reference in its own right for how this node expects segment prompts to be formatted — worth reading even if you don't use Gemini specifically.

None of the above are required for MuseDirectorSamplerV10Final itself to work — only for this specific example graph exactly as saved. A much simpler graph (loaders → this node → Video Combine) is all you actually need to get started.

Timeline tracks

| Track | Purpose | |-------|---------| | MAIN | Image/video segments — the reference frame(s) for generation, and prompt text | | AUDIO | Speech or music clips for Custom Audio mode | | BG AUDIO | Background ambience — layered under everything, volume controlled by bg_volume | | MOTION | Reference video segments for IC-LoRA motion guidance |

Character reference images and descriptions live in the character-card panel at the bottom of the timeline (@ref1/@ref2/@ref3), not on a timeline track.

Audio toggles

Three buttons in the node toolbar control audio behaviour, usable individually or in combination.

Gen Audio (ON only) — LTX generates ambient sound and atmosphere from [SOUNDS] tags in your prompts. No audio file needed.

Custom Audio (ON only) — plays the audio file(s) from the AUDIO timeline track directly in the output, passed through as-is.

Custom Audio + Lipsync (Custom Audio ON, Lipsync ON, Gen Audio OFF) — drives the character's lip movements to match the speech in your custom audio file. Requires a speech file on the AUDIO track and the talking head LoRA loaded.

Gen Audio + Custom Audio (both ON, Lipsync OFF) — generates ambient audio from [SOUNDS] prompts AND plays the custom audio file, mixed together.

BG Audio track is independent of all three toggles — drop any audio file onto it and it mixes under the main output at bg_volume.

Prompt format

Use uppercase tags inside segment and global prompts:

A woman sits at a podcast desk, talking confidently to camera.
[SPEECH] Right. I'm going to tell you something most people in this space won't admit.
[SOUNDS] Quiet studio, soft air conditioning hum, distant city traffic

| Tag | Effect | |-----|--------| | [SPEECH] | The words the character should say — used with lipsync or generated audio | | [SOUNDS] | Ambient sounds and atmosphere — used when Gen Audio is ON | | @ref1 / @ref2 / @ref3 | References a character-card slot. In Ghost Mask (End) mode, invisibly swapped for that slot's analyzed description, and also used to restrict which slot(s) get sent as reference for that chunk (untagged prompts still send every filled slot). In Licon MSR (Prefix), left in the prompt as-is and used the same way to pick which slot(s) feed the identity guide. |

Important: [SPEECH]/[SOUNDS] tags must be uppercase — lowercase is ignored by the model.

Running the test suite

cd custom_nodes/MuseDirectorV10Final
python tests/run_tests.py

No ComfyUI server or GPU needed — these test the reference-image loading, Ghost Mask/MSR guide building, prompt-relay masking, and MSR frame layout as pure Python logic. They catch code-level regressions (a function returning the wrong thing), not "does the output actually look right" — for that, run a real generation.

Tips & known limitations

Ghost Mask reference-image composition matters. A single multi-panel reference sheet (e.g. one image with four angles side-by-side) gets resized/padded as one image, which can visibly hurt identity fidelity. Prefer separate individual images per angle in a character slot over one composite sheet.
reference_strength feeds directly into LTXVAddGuide's (Ghost Mask) or LTXAddVideoICLoRAGuide's (MSR) own strength argument — very high values (2.0+) can produce visible haloing/baked-in artifacts near the guide anchor.
ghost_anchor_buffer (default 2 latent frames) inserts empty latent frames between the real visible content and the first hidden Ghost Mask reference — raise it if you see faint artifacting building up in the last several visible frames of a chunk near the anchor.
enable_ambient_pass is a genuine tradeoff, not a clean fix: the ambient/SFX audio pass sees the actual video as visual context (so it can generate matching sound), which can produce a faint duplicate/echoed voice in generated-audio mode. Turning it off removes that but also removes real ambient sound (waves, crowd noise, etc). Default ON; consider OFF for generated-audio-only testing.
single_stage_mode skips Stage 2 entirely and samples once at full resolution. Mechanically solid, but has no confound-free quality comparison against the normal two-stage flow yet — treat it as experimental.
Keep negative prompts narrow and specific to the actual problem you're seeing, rather than a large generic template — overly broad negative prompts (e.g. unrelated phrases about geometry or unrelated objects) can have unexpected side effects on body/shape rendering.
Use global_prompt for style/setting/lens language that should persist across the whole clip. Keep @ref1/@ref2/@ref3 tags in per-segment prompts rather than switching to plain pronouns once a scene might have more than one character — pronouns can't disambiguate between people the way tags can.

Credits

LTX 2.3 AV model by Lightricks
Licon MSR (Prefix) reference-frame layout (subject/background image prep and latent-aligned frame allocation in _build_msr_guide_data) adapted with credit from liconstudio/ComfyUI-Licon-MSR (MIT License, Copyright (c) 2026 LiconStudio) — the reference implementation for the LTX 2.3 MSR technique
Built by Muse Collective

License

MIT.

muse-director-v10-final

Nodes (3)