IPAdapterAnimate by Chan

IPAdapterAnimate is the frame-by-frame machine behind ComfyUI-IPAnimate, and its whole pitch is one sentence from the README: AnimateDiff output is blurry, so let's not use AnimateDiff at all. Instead of a temporal motion module, you hand it a handful of keyframe images and it drives a sampler through them one frame at a time, crossfading each keyframe's influence with IP-Adapter while a ControlNet holds the structure down. The result is sharper and way more controllable than a motion module - at the cost of being a genuinely fiddly, manual way to make video.

This is the same idea as Steerable Motion, minus the AnimateDiff dependency. Worth knowing up front: IP-Adapter loads nothing newer than SDXL (it's a 2023 Tencent adapter, and the ecosystem has moved on), so this whole approach is an SD 1.5 / SDXL game. If that's your base, it works great.

How it works

You feed in your keyframe images as one batched IMAGE tensor (at least two of them - the code asserts on it), plus your model, conditioning, an IP-Adapter model, and its CLIP vision. The node builds an animation flow: for every pair of adjacent keyframes it generates a stack of frames, each pair expanded by your frame distribution, and for each of those frames it computes a pair of strengths via sine/cosine interpolation between the low and high end of your strength tuple. One keyframe's influence fades out as the next fades in.

Then the index input says which frame of that flow to compute. The node applies IP-Adapter twice - once per keyframe image of the current transition, weighted by that frame's crossfade values - and, if you've wired it, a ControlNet pass too. What comes out is a patched MODEL plus POSITIVE/NEGATIVE ready for a single sampler step. You decode, save, bump index by one, and loop. The NET_INDEX output (always index + 1) exists precisely so you can feed it straight back into index and keep the loop running. LOGS dumps a JSON string showing exactly what was applied to that frame - index, the keyframe pair, and the per-step IP-Adapter/ControlNet strengths. Genuinely handy when a frame comes out wrong.

The inputs that matter

• index - which flow frame to compute this pass. Loop it via NET_INDEX.
• type_of_frame_distribution - linear (every transition gets linear_frame_distribution_value, default 8 frames) or dynamic (dynamic_frame_distribution_values like "16,8,8,16" - per-transition frame counts, so your hold a beat where you want it).
• type_of_strength_distribution - linear uses linear_strength_value ("(0.1,0.9)" by default, min/max of the crossfade) applied to every transition; dynamic takes a list of tuples, "(0.0,1.0),(0.0,1.0),...". Both are parsed with Python eval, so keep the tuple syntax exact.
• relative_ipadapter_strength (1.0) and relative_cn_strength (0.3) - global scales on top of the per-frame curves. Raise the first for stronger reference influence, the second to let ControlNet boss the geometry around more.
• control_net + control_images (optional) - per-frame control images (tile or depth preprocessed from your keyframes work well) and a loaded ControlNet. If either is left unconnected the node silently skips ControlNet and runs IPA-only. You'll want to check LOGS if the output is drifting.

Install

Install the pack once via Manager (search "ComfyUI-IPAnimate") or:

cd ComfyUI/custom_nodes
git clone https://github.com/Chan-0312/ComfyUI-IPAnimate

then restart ComfyUI. There's no requirements.txt - the apply logic is pure torch/PIL/numpy, and the pack bundles its own copies of cubiq's IPAdapter_plus and Kosinkadink's Advanced-ControlNet under imports/. But you still need the loader nodes: ComfyUI_IPAdapter_plus for IPAdapterModelLoader (nothing else produces the IPADAPTER input), plus a ControlNet loader and a CLIPVisionLoader. The demo workflow also leans on VHS Load Images, Inspire Pack's KSampler, and a tile preprocessor - those are workflow extras, not node requirements. You'll want an IP-Adapter Plus model for your base (SDXL needs the matching ip-adapter-plus_sdxl_vit-h + CLIP ViT-H weights) and a ControlNet.

Where people get burned

Because this is per-frame img2img rather than a temporal model, you trade AnimateDiff's blur for flicker - frames drift and jitter across the loop, and nothing is holding temporal coherence except your keyframes and prompts. That's the honest tradeoff baked into the approach. Also: overrun the flow length with a bad index and the assert throws; forget that dynamic values are eval'd strings and a stray space breaks them; and on longer sequences the per-frame IP-Adapter on a single persistent model can climb VRAM, so keep keyframes and swap counts sane.