Apply Ref Motion

This is the whole pack in one node. Apply Ref Motion takes your AnimateDiff model and a reference video's latents and patches the model so the next KSampler pass generates frames that move like the reference. Not the content - the motion: a character's walk cycle, a camera drift, waves. Everything else in MotionThiefExperiment is just building the config this node consumes.

A word up front: the author literally calls this a hack for experiments only - no API, no trained model, nothing to download. The price is that it's finicky, SD 1.5/AnimateDiff-only, and untouched since 2024. Nobody on Reddit is talking about it. Treat it as a fun lab, not a production pipeline.

What it actually does

The trick is attention transfer, and it's the video cousin of ControlNet Reference (which guides a generation with a still image's self-attention features). Here it happens along the temporal axis instead - AnimateDiff's motion module is built on temporal transformer blocks, self-attention across frames, and this node hijacks them.

Looking at the source, each UNet forward inside the active window runs twice. First a WRITE pass: your reference latents are re-noised to the current step's sigma and pushed through the model, and the participating temporal attention layers record their keys, values, and hidden states into "banks". Then the READ pass runs your generation while those same layers attend to the recorded reference banks instead of (or alongside) their own. Your motion gets pulled toward the reference's at every step. The code comment says it plainly: # HACK: This is for experiment purposes only.

The implementation swaps the diffusion model's class for a wrapped UNet and monkeypatches patch_model to wrap every temporal attention block. Because it patches patch_model, it can fight other nodes that do the same, and it only works when the model actually has temporal blocks.

Inputs that matter

• model - your AnimateDiff chain, i.e. after ADE_UseEvolvedSampling in the example workflow. If there's no motion module in the model yet, there are no temporal blocks and this node silently does nothing.
• ref_latents - the reference video's frames VAE-encoded. In the example: VHS_LoadVideo → ImageScale down to 512×512 → VAEEncode. The reference must have at least as many frames as your sample batch or it raises a ValueError.
• start_percent / end_percent - the window of sampling the reference steers. Defaults are 0.0 to 0.7, so the last 30% runs on your generation's own momentum. Widening it pushes harder but risks dragging the reference's content in.
• positive / negative - text conditioning that conditions the reference pass, not your output. The example uses a different prompt here ("explosions") than in the KSampler ("puddle"): you can steer which motion gets stolen.
• ref_settings - the MOTION_REF_SETTINGS object from the settings node. Required - don't forget the wire.
• enabled - off returns the model untouched, a clean bypass for A/B testing.

The one output is a patched MODEL that goes straight into your KSampler.

Installing it

ComfyUI Manager: search "ComfyUI-MotionThiefExperiment" (or just "MotionThief") under Custom Nodes, install, restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/logtd/ComfyUI-MotionThiefExperiment

Restart ComfyUI. There's no requirements.txt and no pip install - the pack imports only torch, einops, and ComfyUI internals. The workflow, however, pulls in real baggage: install ComfyUI-AnimateDiff-Evolved, VideoHelperSuite (for loading/combining video), drop an AnimateDiff motion module like v3_sd15_mm.ckpt into models/motion_modules, and use an SD 1.5 checkpoint (the example uses photon_v1.safetensors).

Where people get burned

• "Motion Reference latent must be at least as long as sample latent" - your reference video has fewer frames than the EmptyLatentImage batch. Load more frames or shrink the batch. The example runs 16 frames, AnimateDiff's native window.
• Nothing happens at all - you fed it a plain checkpoint. Apply it after the AnimateDiff nodes, not before; no temporal blocks, no theft.
• VRAM and time jump - the WRITE pass is a whole extra UNet forward per step inside the window, so expect roughly double the work while it's active. That's why end_percent at 0.7 isn't just aesthetic.
• Content bleeding through - you're replaying the reference's actual attention features, so at high settings the subject's look leaks in. Dial the settings node's fidelity, not the KSampler.

It won't work on SDXL or Flux, the motion module doesn't transfer, and nothing's maintained it for ComfyUI's current API - expect to babysit it. But when it lands, watching your clip pick up the reference's rhythm is satisfying, and it's the cheapest motion-transfer trick the SD 1.5 era left behind.