GetWarpedNoiseFromVideoAnimateDiff

This is the odd one out in Kijai's VideoNoiseWarp pack, and it's worth explaining why before anything else. The whole pack exists to port the "Go with the Flow" trick - warping your starting noise along the optical flow of a real video so the diffusion model begins with motion already baked in. For CogVideoX and HunyuanVideo, that's all you need. For AnimateDiff, there's an extra ingredient: the README points at a motion LoRA by spacepxl (spacepxl/animatediffv3_warpednoise_motionlora) that was trained to expect this kind of noise. That LoRA is the reason the AnimateDiff variant of the node exists at all - without it, warped noise buys you much less on the SD 1.5 stack.

Spacepxl trained that LoRA in a single evening (rank 128, if the thread's worth of detail matters to you) by teaching the AnimateDiff motion module to work with warped-noise inputs. So the intended flow is: load a normal SD 1.5 checkpoint, load that motion LoRA, feed this node's noise output into the sampler as the initial latent, and let the combination produce motion that follows your reference video. It's a niche, experimental pipeline, but it's also the only one of these four nodes with a purpose-built LoRA behind it.

How it works

Same RAFT optical-flow pipeline as the rest of the pack - warps a seeded noise field along the detected motion, frame by frame, keeping the noise statistically gaussian. The difference is in the fixed settings: 4 noise channels (SD's latent count), BCHW latent shape, no temporal interpolation (one noise frame per input frame - AnimateDiff's motion module supplies the temporal structure itself), and an 8x spatial downscale to match SD's VAE.

One quirk distinguishes this node from its siblings: model and sigmas are required sockets here, whereas on the generic node they're optional. Connect them and the noise gets scaled to the sampler's sigma range, which is exactly what you want when the latent goes straight into a KSampler.

The inputs you set

The full list is short - images, degradation, seed, model, sigmas, output_device. You really adjust three:

• images - your reference video frames, the motion you want the noise to inherit.
• degradation (0 to 1, default 0) - blends the warped noise toward fresh random noise. The LoRA was trained on warped noise, so start at 0 and only nudge it up if motion looks too aggressive.
• seed - the initial noise field. Same seed, same noise, same vibe.

Outputs

noise (LATENT - into your sampler) and visualization (IMAGE - a normalized preview of the noise field stretched back to your video's frame count, so you can check the warp actually tracks the motion). Note there's no optical_flows output on this variant: the pack skips computing the flow visualization here, which also makes it a bit faster per frame.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-VideoNoiseWarp

Or search VideoNoiseWarp in ComfyUI Manager. Dependency is just einops; the RAFT optical flow model downloads automatically on first run (one-time, needs internet). Then grab spacepxl/animatediffv3_warpednoise_motionlora from Hugging Face and load it like any other LoRA.

Realistic expectations

Be honest about what this is. AnimateDiff is the SD 1.5-era path, and even the KB's own framing is that its classic advantages have mostly been matched by newer video models - this survives as a specific look and a specific experimental corner, not a general workflow. The pack itself is a marked WORK IN PROGRESS, dormant since spring 2025, and the motion LoRA was trained quickly on a niche paradigm. You're doing this because the Go-with-the-Flow motion feel on an SD 1.5 stack is something you want, not because it's the state of the art. If your reference video is long or high-res, expect the RAFT pass to be slow and VRAM-hungry, and check the visualization output early to confirm the noise is actually following motion before you burn sampling time on it.