ADMD_InitializeTraining

ADMD_InitializeTraining is the node that decides what you're actually training. It takes the PIPELINE from one of the loaders plus your source video frames and a prompt, and does two things: it builds the two LoRA stacks that MotionDirector training is all about, and it runs a sanity check on your input frames so you can confirm the data actually loaded before you burn hours on a run. Its outputs - admd_pipeline and lora_info - are what every ADMD_TrainLora, ADMD_ValidationSampler, and ADMD_SaveLora downstream consumes.

This is also where the lora_name is minted. Everything the run saves is named from it, so pick something meaningful before you start - "motiondirectorlora" (the default) is fine for a test, useless three runs later.

The two LoRA stacks

MotionDirector's trick is separating the problem. It attaches a temporal LoRA to the UNet's TemporalTransformerBlocks (which carry AnimateDiff's across-frame attention) and a spatial LoRA to the Transformer3DModel blocks (the in-frame stuff). Each gets its own optimizer and learning rate. include_resnet (default off) adds ResnetBlock2D to the spatial target list - that's a quality-per-VRAM tradeoff; leave it off until you know you want it. The spatial LoRA also gets a 0.1 dropout and is masked out entirely on ~20% of steps, which is MotionDirector's regularization against it learning the look instead of the motion.

Inputs that matter

• pipeline - from your loader. images - your training video as frames (a 16-frame clip; the pack's workflows only claim support for 16-frame inputs). prompt - describe the action, not the scene: the example workflow uses "car is driving in desert".
• max_train_steps (default 300) - the total budget for the whole run. The example workflow's note is explicit: set this to the combined max of all your ADMD_TrainLora nodes.
• learning_rate (5e-4) and learning_rate_spatial (1e-4) - temporal and spatial LoRA learning rates. Watch this one: when you pick the Lion optimizer (the default), the code divides both by 10 internally, so your 5e-4 becomes 5e-5. That's by design - Lion wants smaller rates - but it means the number you type isn't the number used.
• lora_rank (64) - rank for both LoRAs. 64 is the shipped default and fine for most clips.
• optimization_method - Lion (default), AdamW, or prodigy. Prodigy needs prodigyopt installed or the node raises an ImportError. Lion divides the LR by 10 and scales weight decay by 10 as noted above.
• seed - for reproducible training.

Outputs: sanitycheck (an IMAGE of your input frames, normalized and saved to the pack's own outputs/<date>/<name>/sanity_check/ folder as an mp4 - check it, it's the fastest way to catch a bad frame load), admd_pipeline, and lora_info (the name/rank/paths used later by ADMD_SaveLora).

Install and gotchas

Pack install: Manager → "ADMotionDirector", or clone into custom_nodes/, then the real step:

cd ComfyUI/custom_nodes/ComfyUI-ADMotionDirector
pip install -r requirements.txt

(portable: python_embeded\python.exe -m pip install -r ...). It needs diffusers, transformers, lion-pytorch, peft, loralib, einops, omegaconf, and imageio - the lion-pytorch one is non-negotiable, it's the default optimizer.

Where people get burned: the sanity-check/sample videos are saved by imageio, and if imageio-ffmpeg isn't present you'll hit a confusing error about saving video (the classic 2024 thread trail is an imageio "fps" parameter clash that was fixed by pinning an older imageio - but the pack requires imageio>=2.33.1 plus imageio-ffmpeg>=0.4.7, so first make sure the ffmpeg backend is actually installed). And keep resolution small - the shipped workflows say 256/384/512 are the sizes known to work well. Larger frames train slower and eat VRAM fast.