Nodes/ComfyUI-CustomNodeKit/WanSCAILToVideoMultiRef
ComfyUI Node

WanSCAILToVideoMultiRef

Feed SCAIL-2 several reference images and colored masks, get conditioning back

By user2318·Created 4 months ago·Updated about a month ago· 56
WanSCAILToVideoMultiRef
  • positive
  • negative
  • vae
  • pose_video
  • pose_video_mask
  • reference_image
  • reference_image_mask
  • clip_vision_output
  • previous_frames
  • positive
  • negative
  • latent
  • video_frame_offset
width512
height896
length81
batch_size1
pose_strength1.00
pose_start0.00
pose_end1.00
ref_encoding_mode1+4n批量
video_frame_offset0
previous_frame_count5
replacement_modefalse

SCAIL-2's whole pitch is that it dropped the stick-figure pose representation and takes raw inputs instead: a reference image, a driving video, and support masks. That's more flexible - cross-identity replacement, animal driving, multi-character - but it also means a lot of conditioning engineering before the sampler ever runs. WanSCAILToVideoMultiRef is that engineering in one node: the conditioning hub for this pack's SCAIL/SCAIL-2 support. You throw in reference images, a pose video, colored per-identity masks, and it returns the positive/negative/latent triple plus a frame-offset for chaining chunks.

It's the SCAIL sibling of WanAnimateToVideoCustom, and it does the multi-reference thing properly: every reference image is independently encoded into its own reference_latents (SCAIL-2 trains with reference images concatenated into the latent, so more references means more identity anchors), and the pose video is downscaled to half resolution before encoding.

The inputs that matter

  • positive / negative / vae - conditioning and the VAE (required).
  • width / height (default 512×896, step 32) and length (default 81) - output geometry. 81 is the SCAIL native context; keep it there for single chunks.
  • reference_image - a batch of images (N, H, W, 3). Each is encoded independently and added as a separate reference latent.
  • ref_encoding_mode - 1+4n批量 (first ×1, rest ×4, one batch encode - closest to training, the default), 逐帧编码 (each frame independently encoded, higher per-frame fidelity), or 混合编码 (first N-1 batch + last frame per-frame - a balance that eases chunk-to-chunk jumps).
  • pose_video - the driving video for pose conditioning.
  • pose_video_mask - SCAIL-2 only: a SAM3 colored per-identity mask video at the same resolution as the pose video. This is what makes multi-person work.
  • replacement_mode - False (default) is Animation Mode, where the pose mask has a black background; True is Replacement Mode with a white background. The mask background color must match this setting or you get garbage.
  • reference_image_mask - SCAIL-2 only: colored masks matching the reference images, same count.
  • previous_frames + previous_frame_count (default 5) - the anchoring pair for chunk continuation: feed the previous chunk's decoded output; only the last previous_frame_count frames are used as the extension anchor. 5 is what SCAIL-2 was trained with (81-frame chunks, 76-frame step).
  • video_frame_offset - the chunk counter; wire the previous chunk's offset output into this input.
  • pose_strength (default 1), pose_start / pose_end - how hard and when the pose conditioning applies during denoising.

Outputs

positive / negative (the assembled conditioning, including the 28-channel mask fields), latent (the empty generation-size latent), and video_frame_offset (your offset plus length, for the next chunk). From here the flow is: this node → WanSCAILContextWindows (wraps the model) → WanSCAIL2PhaseSampler or a KSampler → VAE decode → next chunk's previous_frames.

Where people get burned

The mask/background contract is the #1 trap: Animation Mode wants black-background pose masks and produces white reference-mask backgrounds; Replacement Mode flips both. The pack's Create SCAIL-2 Colored Mask node handles this pairing automatically, so use it rather than hand-building masks. Also, multi-reference isn't free - each extra reference adds latent frames and VRAM, and the compute is already the wall with this model. And remember the community's verdict on quality: outputs are genuinely impressive, faces can wander, and 20 seconds of video costs tens of minutes of generation.

Install

Part of ComfyUI-CustomNodeKit. ComfyUI Manager → search "ComfyUI-CustomNodeKit", or:

cd ComfyUI/custom_nodes
git clone https://github.com/user2318/ComfyUI-CustomNodeKit.git
cd ComfyUI-CustomNodeKit
pip install -r requirements.txt

Restart ComfyUI. You supply the SCAIL-2 weights and the SAM3 tracking yourself.

Categorymodel/conditioning/video_models

Inputs (20)

NameTypeDefaultDescription
positiveCONDITIONING
negativeCONDITIONING
vaeVAE
widthINT51232–16384
heightINT89632–16384
lengthINT811–16384
batch_sizeINT11–4096
pose_strengthFLOAT1.000–10姿态 latent 的强度。Strength of the pose latent.
pose_startFLOAT0.000–1姿态 conditioning 的起始步。Start step of the pose conditioning.
pose_endFLOAT1.000–1姿态 conditioning 的结束步。End step of the pose conditioning.
ref_encoding_modeCOMBO1+4n批量参考图编码模式。1+4n批量=第1张×1其余×4后一次编码(默认,更接近训练);逐帧编码=每张图独立编码后时间维拼接(高保真);混合编码=前N-1张1+4n批量+最后1张逐帧编码(兼顾清晰度与减轻段间跳变)。Reference encoding mode: 1+4n batch=1st×1 rest×4 then batch encode (default, closer to training); per-frame=independent encode then temporal concat (high fidelity); hybrid=first N-1 as 1+4n batch + last frame independently.
video_frame_offsetINT00–16384当前块开始的累计输出帧偏移。从上一块的 video_frame_offset 输出接入。Cumulative output frame this chunk begins at. Wire from the previous chunk's video_frame_offset output.
previous_frame_countINT51–16384用于锚定的上一块尾帧数。SCAIL-2 训练使用 5(81帧块,76帧步长)。Tail frames of previous_frames to anchor. SCAIL-2 trained at 5 (81-frame chunks, 76-frame step).
pose_videooptIMAGE用于姿态 conditioning 的视频,将降低分辨率至主视频的一半。Video used for pose conditioning. Will be downscaled to half the resolution of the main video.
pose_video_maskoptIMAGE仅 SCAIL-2。与 pose_video 同分辨率的 SAM3 逐人彩色遮罩视频。SCAIL-2 only. Colored per-identity SAM3 mask video at the same resolution as pose_video.
replacement_modeoptBOOLEANfalse仅 SCAIL-2。False=动画模式(pose_video_mask应为黑色背景);True=替换模式(pose_video_mask应为白色背景)。SCAIL-2 only. False = Animation Mode (black bg). True = Replacement Mode (white bg).
reference_imageoptIMAGE参考图输入,多张参考图以批次形式传入 (N, H, W, 3)。每张图独立编码后作为单独的 reference_latents 添加。Reference image(s). For multiple references, pass a batch of images (N, H, W, 3). Each image is independently encoded and added as a separate reference_latents.
reference_image_maskoptIMAGE仅 SCAIL-2。与 reference_image 同分辨率的彩色参考遮罩。应与 reference_image 数量匹配。SCAIL-2 only. Colored reference mask(s) at the same resolution as reference_image. Should match the reference_image count.
clip_vision_outputoptCLIP_VISION_OUTPUT用于 conditioning 的 CLIP 视觉特征。模型使用拉伸缩放至宽高比进行训练。CLIP vision features for conditioning. Model is trained with stretch resize to aspect ratio.
previous_framesoptIMAGE仅 SCAIL-2。上一块的完整解码输出。仅最后 previous_frame_count 帧用作扩展锚定。SCAIL-2 only. Full decoded output of the previous chunk. Only the last previous_frame_count are used as the extension anchor.

Outputs (4)

NameTypeDescription
positiveCONDITIONING
negativeCONDITIONING
latentLATENT生成尺寸的空 latent 张量。Empty latent of the generation size.
video_frame_offsetINT调整后的偏移量 + 长度。接入下一块。Adjusted offset + length. Wire into the next chunk.