GetWarpedNoiseFromVideoCogVideoX

If you saw the Go-with-the-Flow demos in January 2025 - the bullet tearing through the ice cream, the video that had half of r/StableDiffusion talking - this is the node that made the ComfyUI version of that work. When the technique hit, the community reaction was basically "Kijai, we need you," and Kijai responded with this pack plus an example workflow in his CogVideoXWrapper repo. GetWarpedNoiseFromVideoCogVideoX is the marquee node: a CogVideoX-tuned front end for warped starting noise, and the one the original demo thread actually used.

The trick, in one breath: instead of feeding CogVideoX a random noise latent and hoping it invents camera motion, you feed it noise that has been physically dragged along the optical flow of a real video - measured by a RAFT model between consecutive frames, then warped frame by frame while keeping the noise statistically gaussian. The diffusion model starts with the motion structure already present, and the output is dramatically steadier and more controllable. The reference video's pixels never go into the sampler; only their motion does, encoded as distorted noise.

How it works here

The CogVideoX variant fixes the parameters the model cares about: 16 noise channels (CogVideoX's latent channel count), BCTHW latent shape, and the 4x temporal compression of CogVideoX's VAE - so a 49-frame clip becomes 13 latent frames. num_frames is what you set, and the node computes (num_frames - 1) / 4 + 1 latents for you. It's meant to pair with Kijai's own ComfyUI-CogVideoXWrapper, which is how the demo workflows are wired.

The inputs that matter

• num_frames (default 49) - how many video frames your target video will have. The noise is interpolated to match.
• noise_downtemp_interp - how the warped noise gets collapsed from video-frame count to latent count. nearest is the default and works fine; blend averages neighbors; blend_norm also renormalizes; randn throws the warp away entirely; disabled keeps every frame.
• degradation (default 0.5 here) - blends the warped noise toward fresh random noise. Note the default: unlike the generic node, this variant ships at 0.5, meaning half fresh noise by default. Pure warped noise can over-determine the output and lock your motion to the reference; Kijai's default hedges that.
• seed - the initial noise field. Same video, same seed, same noise.

Outputs

noise (LATENT) goes into your CogVideoX sampler as the initial latent. visualization (IMAGE) is the noise field's first channels normalized to a viewable image, so you can sanity-check that the noise actually follows your motion. And unlike the AnimateDiff and Hunyuan variants, this one keeps an optical_flows output (IMAGE) - the RAFT motion field drawn as an HSV image where hue is direction and brightness is magnitude. If that looks like static, RAFT isn't seeing your motion, and the noise won't carry it either.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-VideoNoiseWarp

Or search VideoNoiseWarp in ComfyUI Manager. Only declared dependency is einops; no manual model downloads - the torchvision RAFT model auto-downloads the first time the node runs (needs internet for that one run). You'll also want ComfyUI-CogVideoXWrapper and a CogVideoX model if you don't have them.

Realistic expectations

The pack is marked WORK IN PROGRESS and hasn't been updated since spring 2025, so treat it as a stable-in-practice experiment, not a maintained library. The original demo was jaw-dropping and the technique is real - but even its fans noted texture issues (the ice cream reads a bit watery after the bullet passes), so temper your expectations. Watch degradation: at 1.0 the effect is fully disabled, and if your output looks like ordinary random noise, that's usually why. And the RAFT pass runs at full video resolution across every frame pair, so long or high-res reference clips are slow and VRAM-hungry - normal, not a malfunction.