Increasing Motion Scale results in more motion but can sometimes give wild results.

1.0 - 1.4 seems to be the range from my testing but feel free to experiment!

control_v1p_sd15_qrcode_monster.safetensors

https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster/resolve/main/control_v1p_sd15_qrcode_monster.safetensors?download=true

On higher VRAM cards you might want to experiment with FILM VFI. (it's slower and more vram heavy)

Depending on scenes, RIFE sometimes introduces artifacts for me.

Hyper-SD15-8steps-lora.safetensors

https://huggingface.co/ByteDance/Hyper-SD/resolve/main/Hyper-SD15-8steps-lora.safetensors?download=true

Any SD1.5 checkpoint should work.

Make sure to match VAE according to the Checkpoint you use. If none is specified use embedded VAE or vae-ft-MSE-840000-ema-pruned

QRCode Controlnet is responsible for the circular motion from the video mask

/ComfyUI/models/clip_vision folder

Download and rename to "CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors" https://huggingface.co/h94/IP-Adapter/resolve/main/models/image_encoder/model.safetensors

Download and rename to "CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors" https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/image_encoder/model.safetensors

To keep the look closer to original images you can increase the weights of ipadapters and/or change to ipadapter Plus model in the Unified Loader node.

Prompts aren't really needed in this workflow - IPAdapter is doing all the prompting.

Negative prompt is ignored at 1.0 Cfg anyways - Thats where the extra inference speed comes from

Keep seed fixed and play with these values to color correct and sharpen final result to your liking

v3_sd15_mm.ckpt

Installable via Manager -> Install Models

or

https://huggingface.co/guoyww/animatediff/resolve/main/v3_sd15_mm.ckpt?download=true

animatediff/v3_sd15_adapter.ckpt

Installable via Manager -> Install Models

or

https://huggingface.co/guoyww/animatediff/resolve/main/v3_sd15_adapter.ckpt?download=true

On higher VRAM cards I suggest you use starting resolution 320x568 or higher. (Longer to process + higher vram usage)

Keep batch_size same unless you know what you're doing - Currently, IPAdapter attention mask values aren't modular and need to be hand adjusted when changing the batch size.

This should be a black and white looping animation. Preferably 12/24/48 etc frame length in square resolution so it combines nicely with ipadapters attention mask.

You can bypass the 3 video outputs on the right to skip the upscaling process until u are happy with the motion/look of the preview. (Keep seed randomized until u are happy with preview then Fix the seed and unbypass)

This might take a while and depending on scenes the upscaling might not be that significant!

This group can be bypassed to skip the extra upscaling with model step. (Right Click Bypass group nodes) or click the bypass icon top right of the group