Animate Anyone Sampler

If you've loaded one of the pack's example workflows and stared at the graph, this is the node doing all the actual work. Everything else in ComfyUI-AnimateAnyone-Evolved - the three loaders, the pose encoder, the CLIP vision and VAE nodes - exists to feed this one box. It's the sampler, and it's where your character animation actually gets rendered, frame by frame.

What it's for

AnimateAnyone is a pose-to-video pipeline: you give it one reference image of a character and a sequence of pose skeletons (OpenPose frames extracted from a video), and it animates that character to follow the poses. The whole approach lives in the SD 1.5 world - it layers a temporal attention module on the UNet the same way AnimateDiff does, plus a reference net (functionally an IP-Adapter) and a lightweight pose guider. The author describes it as "unClipModel + AnimateDiff + a light-weight ControlNet-style pose guider + reference net," all trained end-to-end. This node runs that combined beast through its denoising loop.

How it works

You hand it five pre-baked inputs: the reference_unet (character appearance) and denoising_unet (motion-capable UNet) from the two loaders, the ref_image_latent and clip_image_embeds from VAE-encoding and CLIP-encoding your reference image, and the pose_latent from the Pose Guider Encode node. It then builds a diffusers scheduler from sampler_scheduler_pairs plus the beta/prediction settings, and runs the AADiffusion loop with a sliding context window over your frames.

Three settings actually matter on a first run. sampler_scheduler_pairs picks the combo - the example workflow ships with DDIM, and DPM++ 2M Karras is a solid faster alternative (407s vs 426s on a 3080 for the same 24 frames in the author's own benchmarks). steps defaults to 20; more steps fix the background noise glitches people complain about. And context_frames is your VRAM lever: it controls how many frames the temporal attention sees at once, and it's what sets GPU usage, not the total clip length. The author has run 120+ frame sequences on a 3080 by keeping context_frames=24 (or 12) with context_overlap=4.

The delta slider is the subtle one. It's residual-CFG scaling - noise_pred = delta * uncond + cfg * (cond - delta * uncond). At delta=1 it's standard CFG. The author experimented with StreamDiffusion's residual-CFG trick for a speedup but shelved it, so unless you're tuning, leave it at 1.

The optional use_lora / lora_name pair exists for dropping an LCM LoRA on the denoising UNet. Fair warning from the README: stock SD 1.5 LCM LoRAs don't work well here because the model was trained a long way off the base checkpoint.

Output

It outputs a single latent (a LATENT tensor of shape (frames, 4, h, w)). Wire it straight into a VAEDecode, then a VHS_VideoCombine (or SaveImage) and you have a video.

Installing it

This node ships in the ComfyUI-AnimateAnyone-Evolved pack, so the install is the pack install. ComfyUI Manager finds it if you search "AnimateAnyone Evolved", or:

cd ComfyUI/custom_nodes
git clone https://github.com/MrForExample/ComfyUI-AnimateAnyone-Evolved
pip install -r requirements.txt
# if you hit diffusers errors:
pip install --force-reinstall diffusers>=0.26.1

Restart ComfyUI after. The heavy part is the model downloads - the four *.pth weights plus an SD 1.5 UNet folder all go under the pack's pretrained_weights/, and you need a CLIP vision encoder and a VAE in their usual ComfyUI folders. See the README for the exact layout.

The honest part

This is a 2024-era pack (last commit February 2024) running a research-model pipeline. It's slow - a 3080 takes around seven minutes for 24 frames at steps 20, and one r/comfyui user reported roughly six hours on a 3060 Ti before the author's StreamDiffusion speedups landed. It also expects everything scaled to 768x512, the resolution the model was trained at, which is why the example workflows put an ImageScale node on both the reference and the pose video. If a loaded workflow complains about missing nodes or an import failure, the console will tell you which dependency to pip-install - the pack's requirements list is long, and diffusers is the one that bites most often.