SD-CN Animation Vid2Vid

This is the node that made the original SD-CN-Animation famous in the first place, and it's the reason this pack still has a reason to exist in the Wan/LTX era. Feed it any video - a dance clip, a drone shot, a scene from a film - and it re-renders it through whatever Stable Diffusion checkpoint you load, while keeping the frames glued together. Where plain img2img-on-every-frame gives you the classic strobe-flicker mess, Vid2Vid uses optical flow to know where each pixel should be in the next frame, so the style carries through instead of resetting every 1/24th of a second.

How it works

The motion backbone here is RAFT (the "Recurrent All-Pairs Field Transforms" optical flow model from Princeton - the ~20MB raft-things.pth weights). Per frame:

1. RAFT estimates bidirectional optical flow between consecutive input frames. Where forward flow disagrees with backward flow, the algorithm knows a region is occluded - something got covered or revealed.
2. The previous styled frame is warped forward along the flow, so the style travels with the motion instead of being re-invented.
3. The warped result is blended with the current input frame (controlled by blend_alpha), then occlusion areas are inpainted by SD at processing_strength.
4. A low-denoise fix_frame_strength refine pass cleans up, and a histogram match against the input frame locks the colors.

It's the same two-pass philosophy as Txt2Vid, but grounded in real motion estimates from actual footage instead of a model's guess about the next frame.

Inputs that matter

Mostly the same KSampler set as Txt2Vid (model, vae, positive/negative, steps, cfg, sampler/scheduler, seed), plus raft_model from a Load RAFT Model node and frames from a video loader like VHS_LoadVideo.

The Vid2Vid-specific dials, which are worth understanding because they're what stop ghosting:

• processing_strength - main stylization denoise. The README's sweet spot is 0.75–0.85.
• fix_frame_strength - 0.1–0.2 refine pass, or 0 to skip.
• blend_alpha - the warped-styled-vs-original-input blend. Higher keeps more of the original video's structure; lower leans on the stylized warp.
• occlusion_mask_blur - smooths the mask that decides what gets regenerated. 2–4 is the starting range.
• occlusion_mask_trailing - blend the current occlusion mask with the previous frame's to cut ghosting. Enable it if moving subjects leave afterimages.
• The three occlusion_mask_*_multiplier knobs weight the flow-consistency vs. frame-difference signals that build the mask; leave them alone until you're debugging.

Optional control_net + cn_strength: this one is a genuine highlight. Unlike most video stylization setups, the ControlNet hint is the current input frame, automatically, every frame. A depth or lineart ControlNet preserves structure, so your style lands on the footage rather than replacing it.

Output is the styled frames IMAGE batch → VHS_VideoCombine.

Install

ComfyUI Manager (search "ComfyUI-SD-CN-Animation"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/pxl-pshr/ComfyUI-SD-CN-Animation

Restart. Deps are opencv-python, scikit-image, scipy; the ~20MB raft-things.pth auto-downloads from Hugging Face into ComfyUI/models/RAFT/ on first launch.

Gotchas

You need at least 2 input frames or the node refuses with a clear error. The algorithm's known weakness - the original author admitted it in 2023 - is ghosting and occasional flicker on fast, complex motion; that's what occlusion_mask_trailing and the mask multipliers exist to fight. Also manage your expectations on speed: two sampling passes per frame means a 30-second clip at 24fps is a serious session, even with steps dropped to 10–12. And if you're just after a quick "cartoon-ify my video," a per-frame img2img with a good ControlNet is faster; this node is for when the flicker drives you up a wall.