Nodes/ComfyUI-MagicAnimate/Load Magic Animate Model
ComfyUI Node

Load Magic Animate Model

This Loader Drags 15GB of Models Onto Your Disk (and That's the Good Version)

By thecooltechguy·Created 3 years ago·Updated 2 years ago· 225
Load Magic Animate Model
    • MAGIC_ANIMATE_MODEL
    controlnetappearance_encoder/diffusion_pytorch_model.safetensors
    appearance_encoderappearance_encoder/diffusion_pytorch_model.safetensors
    motion_moduleappearance_encoder/diffusion_pytorch_model.safetensors
    device

    Most ComfyUI loaders pull one file from disk. This one assembles an entire research pipeline. MagicAnimateModelLoader (shown in the menu as "Load Magic Animate Model") is the front half of the ComfyUI-MagicAnimate pack, and it explains in one glance why the README warns you up front: this pack auto-downloads ~15GB of model weights on first launch. Earlier versions dragged down far more - people reported ~100GB before the author optimized it. The good news: it happens once, via the pack's prestartup script, straight into ComfyUI/models/MagicAnimate. The loader then exists to turn those files into a working pipeline.

    What it's actually loading

    MagicAnimate is a late-2023 research model from NUS/ByteDance that animates a still photo of a person using a DensePose driving video - the pose from one clip, the identity from your image. It doesn't fit ComfyUI's usual "checkpoint + VAE + ControlNet" pattern, because it needs a fixed stack of SD 1.5 components plus two custom networks. This node builds that stack, and it's a big one:

    • a full SD 1.5 UNet with a temporal attention motion module injected (the same trick AnimateDiff popularized),
    • an appearance encoder that captures the identity from your source image,
    • a DensePose ControlNet that steers the body geometry from the driving video,
    • the sd-vae-ft-mse VAE, a CLIP text encoder/tokenizer, and a DDIM scheduler.

    Everything gets cast to fp16 and loaded onto your device. The output is a single MAGIC_ANIMATE_MODEL bundle that feeds straight into the MagicAnimate node's magic_animate_model input. There's nothing else it wires into.

    The inputs that matter

    The node exposes four required inputs, and three of them are a trap in disguise.

    • controlnet, appearance_encoder, motion_module - three dropdowns, and each one lists the same seven files from your MagicAnimate folder. Worse, the default for all three points at the appearance encoder. If you just hit Run, the loader will try to build a ControlNet out of an appearance encoder and fall over. Set them properly: appearance_encoder/diffusion_pytorch_model.safetensors, densepose_controlnet/diffusion_pytorch_model.safetensors, and temporal_attention/temporal_attention.ckpt.
    • device - cuda or cpu. The CPU option technically exists. You do not want the CPU option.

    Installing it

    Via ComfyUI Manager, search "ComfyUI-MagicAnimate" and install. Manually, it's the usual dance:

    cd ComfyUI/custom_nodes/
    git clone https://github.com/thecooltechguy/ComfyUI-MagicAnimate
    cd ComfyUI-MagicAnimate/
    python -m pip install -r requirements.txt
    

    The requirements.txt is light - diffusers, omegaconf, torchvision, Pillow - but the pack's Python imports also lean on einops and accelerate, which normally arrive with ComfyUI's own environment. The heavy lift is the 15GB of weights, which download automatically on the next ComfyUI start (you'll see "Downloading Magic Animate's appearance encoder..." lines in the console). Don't interrupt that mid-run.

    Where people get burned

    • Nodes silently missing from your menu. The pack's __init__.py hard-asserts that at least one file exists in models/MagicAnimate. If the download failed or you moved the folder, the whole pack fails to load and ComfyUI just doesn't show the nodes. Check the console - the assert message tells you exactly what's wrong.
    • Re-running the loader. Every execution deletes the previous models and runs gc.collect() before rebuilding the whole pipeline. That's tens of seconds of churn and a spike in VRAM usage. Load once at the start of your graph and leave it.
    • VRAM. You're holding an SD 1.5 UNet, a motion module, an appearance encoder, a ControlNet and a VAE in fp16 simultaneously. This is one of the heaviest single loads in most ComfyUI graphs - not something a laptop iGPU will enjoy.

    One honest caveat: this pack is a point-in-time port (released December 2023, effectively dormant since early 2024). It works, but don't expect active fixes. For what it does, it's remarkably self-contained - all the inference code is bundled in libs/magicanimate, so there are no hidden dependencies to chase.

    CategoryComfyUI Magic Animate

    Inputs (4)

    NameTypeDefaultDescription
    controlnetCOMBOappearance_encoder/diffusion_pytorch_model.safetensors7 options: appearance_encoder/diffusion_pytorch_model.safetensors, control_v11p_sd15_openpose/diffusion_pytorch_model.safetensors, densepose_controlnet/diffusion_pytorch_model.safetensors, sd-vae-ft-mse/diffusion_pytorch_model.safetensors, stable-diffusion-v1-5/text_encoder/model.safetensors, stable-diffusion-v1-5/unet/diffusion_pytorch_model.bin, +1
    appearance_encoderCOMBOappearance_encoder/diffusion_pytorch_model.safetensors7 options: appearance_encoder/diffusion_pytorch_model.safetensors, control_v11p_sd15_openpose/diffusion_pytorch_model.safetensors, densepose_controlnet/diffusion_pytorch_model.safetensors, sd-vae-ft-mse/diffusion_pytorch_model.safetensors, stable-diffusion-v1-5/text_encoder/model.safetensors, stable-diffusion-v1-5/unet/diffusion_pytorch_model.bin, +1
    motion_moduleCOMBOappearance_encoder/diffusion_pytorch_model.safetensors7 options: appearance_encoder/diffusion_pytorch_model.safetensors, control_v11p_sd15_openpose/diffusion_pytorch_model.safetensors, densepose_controlnet/diffusion_pytorch_model.safetensors, sd-vae-ft-mse/diffusion_pytorch_model.safetensors, stable-diffusion-v1-5/text_encoder/model.safetensors, stable-diffusion-v1-5/unet/diffusion_pytorch_model.bin, +1
    deviceCOMBO2 options: cuda, cpu

    Outputs (1)

    NameTypeDescription
    MAGIC_ANIMATE_MODELMAGIC_ANIMATE_MODEL