ComfyUI Node

ADMD_CheckpointLoader

The trainer's front door — feed it any SD 1.5 checkpoint and get a training pipeline

By kijai·Created 3 years ago·Updated 2 years ago· 187
ADMD_CheckpointLoader
  • additional_models
  • PIPELINE
ckpt_name
schedulerDDIMScheduler
use_xformersfalse

ADMD_CheckpointLoader is where a training run actually starts. Give it the ADDITIONAL_MODELS output from ADMD_AdditionalModelSelect plus a checkpoint you already have in ComfyUI/models/checkpoints/, and it converts that familiar .safetensors or .ckpt into the internal PIPELINE object that every training node downstream expects.

This is the "I have a normal CivitAI-style SD 1.5 checkpoint" path - the one the example workflows use (they ship with photon_v1.safetensors selected). If you'd rather load from a HuggingFace-style diffusers folder, or reuse models already loaded elsewhere in your ComfyUI graph, the pack has sibling nodes for those; this one is for the checkpoint you probably already own.

How it works

Under the hood it's a quiet conversion, not a load. The node reads the checkpoint into a state dict, then single_file_utils reshapes it into the diffusers-style components the trainer expects: a VAE, a CLIP text encoder (from the openai/clip-vit-large-patch14 tokenizer), and - the key part - a 3D UNet (UNet3DConditionModel with AnimateDiff's temporal layers bolted on). Then it freezes the UNet, VAE, and text encoder, enables gradient checkpointing, and loads your motion module and domain adapter on top. The frozen base plus the motion module becomes the "validation pipeline" used to preview your progress; the frozen base alone is what your LoRAs train onto.

Two noise schedulers get built alongside, because MotionDirector trains spatial and temporal LoRAs against different schedules: a linear DDIM/DDPM scheduler and a scaled_linear one. You don't touch those; they're carried inside the pipeline.

Inputs and output

  • additional_models - from ADMD_AdditionalModelSelect.
  • ckpt_name - dropdown of everything in models/checkpoints/. SD 1.5 only: AnimateDiff motion modules don't transfer to SDXL or Flux, so don't point this at a big XL checkpoint and wonder why nothing works.
  • scheduler - DDIMScheduler (default) or DDPMScheduler, used for the training noise schedule. Default is fine.
  • use_xformers - off by default. Only has any effect if xformers is actually installed in your environment; it toggles memory-efficient attention on the 3D UNet.

Output is a single PIPELINE, straight into ADMD_InitializeTraining.

Install and gotchas

Manager → search "ADMotionDirector", or clone and restart. The real step is the dependencies:

cd ComfyUI/custom_nodes/ComfyUI-ADMotionDirector
pip install -r requirements.txt

(portable: python_embeded\python.exe -m pip install -r ...). This installs diffusers, transformers, lion-pytorch, peft, loralib, einops, omegaconf, imageio, and friends. Two things bite people here. First, requirements.txt doesn't pin a CUDA-enabled PyTorch - if your environment somehow has CPU torch, this pack's whole class of nodes will be slow or broken; make sure your ComfyUI env has the CUDA wheel yourself. Second, the pack's own README only claims support for PyTorch 2.1.1+cu121 / 2.2.0+cu121.

One behavioral quirk: this node's IS_CHANGED returns nan, which means ComfyUI re-executes it on every queue, no matter how stale the upstream data. That's intentional for a training graph - you want a fresh pipeline each run - but it means the conversion (and the checkpoint read) happens every single time you hit execute. Don't be alarmed by the repeated load; it's the design.

CategoryAD_MotionDirector

Inputs (4)

NameTypeDefaultDescription
additional_modelsADDITIONAL_MODELS
ckpt_nameCOMBO0 options:
schedulerCOMBODDIMScheduler2 options: DDIMScheduler, DDPMScheduler
use_xformersBOOLEANfalse

Outputs (1)

NameTypeDescription
PIPELINEPIPELINE