ComfyUI Node

Wan Director

A video editing timeline inside a single ComfyUI node

By Code2Collapse·Created 6 months ago·Updated a day ago· 52
Wan Director
  • model
  • clip
  • vae
  • clip_vision
  • optional_latent
  • control_video
  • control_mask
  • wan_model
  • wan_t5
  • t5gemma
  • model
  • positive
  • negative
  • video_latent
  • frame_rate
  • combined_audio
  • reference_image
  • info
  • wan_model
  • wan_text_embeds
  • tracks_program
  • control_video
  • control_mask
  • guide_data
  • quality_recipe
backendnative
model_variantwan2.1_i2v
duration_frames81
duration_seconds5.00
frame_rate16
global_prompt
timeline_data
local_prompts
negative_prompts
segment_lengths
guide_strength
display_modeseconds
custom_width832
custom_height480
resize_methodmaintain aspect ratio
cfg_high_noise3.5
cfg_low_noise3.5
ref_strength1.00
everanimate_stagestage2_480p
everanimate_num_chunks1
everanimate_overlap_frames4
everanimate_lora_strength1.00
everanimate_anchor_strategyauto
audio_targetmusic_44k_stereo
enable_prompt_relaytrue
prompt_relay_epsilon0.0010
enable_dynamic_cfgfalse
guidance_rescale_phi0.00
pag_scale0.0
enable_phase_shiftfalse
phase_shift_pct0.70
vae_fp32_decodetrue
enable_multi_clipfalse
structure_prompt
detail_prompt
enable_nagfalse
nag_scale11.0
enable_asymflowfalse
asymflow_shift3.0
cache_typenone
cache_threshold0.10
enable_slgfalse
slg_layers
slg_scale0.70
enable_fetafalse
feta_scale0.50
enable_riflexfalse
riflex_k2

Every Wan video workflow starts the same way: one prompt, one clip, one flat 5-second shot. Fine for a demo. The moment you want a real video - multiple shots, different prompts per segment, audio that lines up - stock nodes leave you splicing separate generations together, or cramming everything into one global prompt and watching the model blend a "close-up of a cat" into a "city skyline" into something that is neither.

Wan Director is the answer that node-level ComfyUI has been missing: a visual timeline, rendered inside the node itself, where you drag image, text, and audio clips onto a track and it hands back the whole conditioning/latent/fps/audio bundle the sampler needs. It's the Wan-native cousin of LTX Director (the author credits WhatDreamsCost's design), rebuilt around Wan's 3D VAE - 16 channels, ÷8 spatial, ÷4 temporal - and it leans on Wan 2.1/2.2 still being the open-video base: Apache 2.0, frozen at 2.2, and the community's default for image-to-video from a strong still.

Honest framing up front: this is a young node from a fast-moving pack with essentially zero public reputation yet - no Reddit chatter, no hype train. You're an early adopter, and if that's fine, this is genuinely the most capable Wan timeline tool I've seen in a single node.

How it works

The heavy lifting is split between a JS extension and the Python node. The JS renders a dual-track timeline (image/text, audio): drag to move or resize segments, right-click to add, drop files to upload, and it serializes everything into hidden JSON string widgets (timeline_data, local_prompts, segment_lengths) that the node parses on execute.

From there it assembles the clip: it builds the Wan empty latent at the right shape, VAE-encodes your first image clip into frame 0 for i2v variants, mixes the audio segments into one waveform, and - the interesting bit - applies PromptRelay, which patches the model's cross-attention so each segment's prompt only steers its own span of frames. With 2+ clips, per-clip prompts become local prompts and global_prompt is the anchor; with a single clip it's a no-op and behaves like a flat prompt. A stack of quality toggles - dynamic CFG, PAG, NAG, AsymFlow, SLG, FETA, RIFLEx, teacache/magcache/easycache - patches the MODEL and emits a quality_recipe JSON for the sampler.

The inputs that matter

Most of the knob farm is advanced and collapsed by default. What you actually set:

  • model_variant - the big one. Eight choices covering wan2.1_t2v / wan2.1_i2v, wan2.2_t2v / wan2.2_i2v, wan_fun_inp / wan_fun_control, wan_animate, and wan2.2_animate_everanimate. It gates which sliders appear and how the latent/conditioning are assembled, so pick it first.
  • backend - native (ComfyUI's built-in Wan; connect model + clip) or kijai (Kijai's ComfyUI-WanVideoWrapper; connect wan_model + wan_t5).
  • duration_frames and frame_rate - 81 frames ≈ 5 s at 16 fps, which is what Wan 2.x trains at. Raise the frame rate for slow-motion-ish output.
  • global_prompt - persistent context prepended to every per-clip prompt.
  • enable_prompt_relay - on by default; it's the whole point of the timeline. Turn it off to encode one flat prompt.

Outputs and wiring

The native branch emits model (patched), positive/negative CONDITIONING, video_latent, frame_rate, combined_audio, reference_image, and an info JSON with warnings. Wire model + positive/negative + video_latent into the Wan sampler, frame_rate into its FPS and the saver, and combined_audio into a save node that takes AUDIO. The kijai branch instead populates wan_model and wan_text_embeds - feed those straight into WanVideoSampler.

Install

It ships in the umbrella ComfyUI-CustomNodePacks pack:

cd ComfyUI/custom_nodes
git clone https://github.com/Code2Collapse/ComfyUI-CustomNodePacks.git

or ComfyUI Manager → search "CustomNodePacks" → install, then restart. The node downloads no weights - bring your own Wan checkpoint and UMT5 via your normal loaders. Two extras: the kijai backend needs ComfyUI-WanVideoWrapper installed separately, and the EverAnimate variant expects an everanimate_<stage>.safetensors LoRA in your loras folder.

Gotchas

  • Mixed sockets - native requires model + clip; kijai requires wan_model + wan_t5. Get it wrong and the node raises a clear error naming what's missing.
  • vae_fp32_decode is on by default - Wan's VAE genuinely looks better in fp32, but it costs VRAM during decode. Turn it off if you're bumping the wall.
  • Variant-specific sliders are ignored elsewhere - the 2.2 dual-CFG sliders (cfg_high_noise/cfg_low_noise) only apply to 2.2 variants, ref_strength only to Animate, the everanimate_* group only to the EverAnimate variant. Setting them on the wrong variant does nothing, despite what the tooltips say.
  • The README is stale - the pack has grown well past its MaskEditControl roots and its README doesn't even mention this node. Trust the in-node tooltips and the auto-generated node reference over the marketing prose.
  • Wan 2.2 speed LoRAs are a separate caution: the community consensus is they can wreck composition and motion, so use them sparingly - low-noise pass only.

If you're assembling multi-shot Wan videos with audio and per-segment prompts, there is nothing else that does this in a single node. Expect rough edges, read the info output, and keep your expectations calibrated - but the design is right.

CategoryC2C/Wan_Director

Inputs (58)

NameTypeDefaultDescription
backendCOMBOnativeWhich video-model stack to drive. native — ComfyUI's built-in Wan implementation. Connect `model` + `clip`. kijai — Kijai's ComfyUI-WanVideoWrapper. Connect `wan_model` + `wan_t5` (optional sockets). PromptRelay (if enabled) is applied to whichever backbone is active and falls back to the generic-introspection patcher for any third-party model.
model_variantCOMBOwan2.1_i2vWhich Wan family / mode this timeline targets. Changes which optional sliders are visible and how the latent + conditioning are assembled.
duration_framesINT811–10000Total timeline length in pixel-space frames. Wan 2.x defaults to 81 frames (≈ 5 s @ 16 fps).
duration_secondsFLOAT5.000.1–1000Total timeline duration in seconds (synced from frames by the UI).
frame_rateFLOAT161–240FPS. Wan 2.x is trained at 16 fps; raise for slow-motion-like output.
global_promptSTRINGPersistent context prepended to every per-clip prompt (characters, lighting, style anchors).
timeline_dataSTRING
local_promptsSTRING
negative_promptsSTRING
segment_lengthsSTRING
guide_strengthSTRING
display_modeCOMBOseconds2 options: seconds, frames
custom_widthINT8320–8192Target width. 0 = inherit from first image clip.
custom_heightINT4800–8192Target height. 0 = inherit from first image clip.
resize_methodCOMBOmaintain aspect ratio4 options: maintain aspect ratio, stretch to fit, pad, crop
cfg_high_noiseFLOAT3.50–20Wan 2.2 high-noise expert CFG. Ignored for non-2.2 variants.
cfg_low_noiseFLOAT3.50–20Wan 2.2 low-noise expert CFG. Ignored for non-2.2 variants.
ref_strengthFLOAT1.000–2Wan Animate reference-image influence. Ignored for other variants.
everanimate_stageCOMBOstage2_480pWhich EverAnimate LoRA checkpoint to apply on top of Wan2.2-Animate-14B: stage1_480p — base motion fidelity (480p training). stage2_480p — Restorative Flow Matching, sharper temporal coherence (recommended). stage3_720p_beta — 720p beta with higher detail; needs more VRAM. Ignored for non-EverAnimate variants.
everanimate_num_chunksINT11–50Long-horizon chunk count. 1 = single ~5 s clip (standard Wan2.2-Animate). ≥2 enables EverAnimate's Persistent Latent Propagation across anchor frames for minute-scale animation. Ignored for non-EverAnimate variants.
everanimate_overlap_framesINT40–16Frames of latent overlap between consecutive chunks (anchor padding). Higher = smoother seams but slower. Ignored if num_chunks=1 or non-EverAnimate variant.
everanimate_lora_strengthFLOAT1.000–2EverAnimate rank-32 LoRA strength. 1.0 = paper default. Ignored for non-EverAnimate variants.
everanimate_anchor_strategyCOMBOautoAnchor-frame selection for chunks 2+: auto — first chunk uses first frame only, later chunks use first + 3 random. first_only — always 1 anchor (faster, slight quality loss). first_plus_random_3 — always 4 anchors (paper-default; best quality). Ignored for non-EverAnimate variants.
audio_targetCOMBOmusic_44k_stereoOutput AUDIO format. Use `speech_16k_mono` if you intend to feed Wan-S2V or any speech-driven pipeline downstream.
enable_prompt_relayBOOLEANtrueInternal PromptRelay: bias each backbone cross-attention block so the timeline's per-clip prompts only steer their own frame span. On by default — the whole point of the timeline. Works on native ComfyUI MODEL, Kijai WANVIDEOMODEL, and arbitrary video-diffusion models (auto-falls back to generic introspection). With 2+ text/image clips the per-clip prompts become the local prompts and `global_prompt` is the anchor; with 0 or 1 clip it is a no-op. Turn off to encode one flat prompt for the whole clip.
prompt_relay_epsilonFLOAT0.00100.000001–0.99PromptRelay penalty decay. <0.1 = sharp boundaries; ≥0.5 softer.
enable_dynamic_cfgBOOLEANfalseCosine-ramped dynamic CFG across denoising steps. Early steps get 1.2× CFG (stronger structure), late steps get 0.7× (softer detail). Prevents oversaturation and improves quality.
guidance_rescale_phiFLOAT0.000–1Guidance rescale (phi). Rescales guided output to match conditional std-deviation, preventing color oversaturation at high CFG. 0=off, 0.7=recommended for Wan 2.2. Requires enable_dynamic_cfg=True.
pag_scaleFLOAT0.00–5Perturbed Attention Guidance scale. Improves prompt adherence by guiding away from identity-attention outputs. 0=off, 1.0–3.0 typical.
enable_phase_shiftBOOLEANfalsePhase-shift sampling: Euler for early steps (structure), DPM++ 2M for late steps (detail). Uses smooth sigma crossfade.
phase_shift_pctFLOAT0.700.3–0.95Step fraction where phase-shift transitions from Euler to DPM++.
vae_fp32_decodeBOOLEANtrueForce VAE decode in fp32 for maximum quality. Wan VAE produces significantly better results in fp32 (recommended by HuggingFace). Uses more VRAM during decode only.
enable_multi_clipBOOLEANfalseMulti-slot CLIP conditioning. Split prompts into structure (early) and detail (late) phases for finer control over generation.
structure_promptSTRINGStructure prompt (active during early denoising, 0–35%). Focus on composition, layout, camera angles, scene description. Only used when enable_multi_clip=True.
detail_promptSTRINGDetail prompt (active during late denoising, 55–100%). Focus on textures, materials, lighting, color grading. Only used when enable_multi_clip=True.
enable_nagBOOLEANfalseNormalized Attention Guidance: boosts prompt adherence via attention-space CFG.
nag_scaleFLOAT11.00–30NAG guidance scale. Higher = stronger guidance.
enable_asymflowBOOLEANfalseAsymFlow time-shift for improved temporal consistency.
asymflow_shiftFLOAT3.00.1–20AsymFlow shift parameter.
cache_typeCOMBOnoneInference caching strategy. Speeds up generation by skipping redundant transformer passes.
cache_thresholdFLOAT0.100–1Cache skip threshold. Lower = more aggressive caching (faster but less accurate).
enable_slgBOOLEANfalseSkip-Layer Guidance: run a second pass with layers removed for quality boost.
slg_layersSTRINGComma-separated layer indices to skip (e.g. '7,8,9'). Empty = auto-select.
slg_scaleFLOAT0.700–2SLG guidance scale.
enable_fetaBOOLEANfalseFrequency-Enhanced Temporal Attention for better frame coherence.
feta_scaleFLOAT0.500–2FETA scale. 0 = off.
enable_riflexBOOLEANfalseRIFLEx RoPE rescaling for length extrapolation beyond training length.
riflex_kINT21–8Number of lowest RoPE frequencies to rescale.
modeloptMODELNative Wan MODEL (2.1 / 2.2 / Fun / Animate). Required when backend='native'.
clipoptCLIPText encoder paired with the Wan model (UMT5 for 2.x). Required when backend='native'.
vaeoptVAEWan VAE. If connected, the Director encodes reference images for i2v and forces fp32 when vae_fp32_decode=True.
clip_visionoptCLIP_VISIONCLIP Vision model for image embeddings (Kijai i2v/Animate). If not connected, image_embeds output is None.
optional_latentoptLATENTOverride the auto-built empty latent.
control_videooptIMAGEFor Wan Fun / Animate: control sequence (depth/pose/canny). Passed through to control_video output.
control_maskoptMASKFor Wan Fun Inpaint: per-frame mask track. Passed through to control_mask output.
wan_modeloptWANVIDEOMODELKijai WanVideoWrapper model patcher (required when backend='kijai').
wan_t5optWANTEXTENCODERKijai T5 text encoder (required when backend='kijai').
t5gemmaoptT5GEMMA_ENCODERR&D: connect a 'T5Gemma Encoder Loader (WNE)' to encode the Director's composed prompt through T5Gemma instead of CLIP (backend='native'). NOTE: T5Gemma hidden states are a different space/width from Wan's UMT5-XXL — this only works on a Wan model finetuned/adapted for T5Gemma; a stock Wan checkpoint will error or produce garbage.

Outputs (15)

NameTypeDescription
modelMODELNative MODEL (patched with PromptRelay + NAG + PAG + Dynamic CFG + AsymFlow as enabled). When backend='kijai', passthrough of input `model` if connected.
positiveCONDITIONINGPositive CONDITIONING (native branch) with guide_strength embedded. Empty list when backend='kijai'.
negativeCONDITIONINGNegative CONDITIONING (native branch). Empty list when backend='kijai'.
video_latentLATENTWan latent: VAE-encoded reference for i2v (if VAE connected) or empty latent. Channels=16, /8 spatial, /4 temporal.
frame_rateFLOATFrame rate echoed for downstream sampler/saver nodes.
combined_audioAUDIOAudio waveform mixed from the timeline's audio segments.
reference_imageIMAGEReference image (first image clip) — used by Wan I2V/Animate as the start/reference frame. Black image if none.
infoSTRINGJSON: resolved backend, variant, latent shape, segment count, audio sample rate, prompt-relay status, quality stack status, warnings.
wan_modelWANVIDEOMODELKijai WANVIDEOMODEL (only populated when backend='kijai'; PromptRelay-patched in place if enabled).
wan_text_embedsWANVIDEOTEXTEMBEDSKijai WANVIDEOTEXTEMBEDS dict (only populated when backend='kijai'). Feed directly into WanVideoSampler.
tracks_programSTRINGJSON: timeline schema_version + normalised lora/camera/seed/pose tracks for downstream applier nodes.
control_videoIMAGEControl video passthrough (IMAGE) for Wan Fun/Animate. None if not connected.
control_maskMASKControl mask passthrough (MASK) for Wan Fun Inpaint. None if not connected.
guide_dataSTRINGJSON: per-segment guide_strength values for downstream guide applier nodes.
quality_recipeSTRINGJSON: quality recipe config (SLG, FETA, RIFLEx, cache, FreeInit, phase-shift) for downstream sampler.