AetherScale • Motion Analysis

This node is the precompute step nobody talks about in the AetherScale DLSS 5 experiments. Neural rendering that stays stable over time needs to know two things: how each frame moved relative to the one before it, and where one shot ends and the next begins. Feed Motion Analysis a frame batch and it measures both, packaging them into a motion packet that the Neural Rendering node can consume as a temporal guide. It's in the pack's AetherScale/Neural category, and in the README's quick-start it's literally step one: frames in, motion out, then let Neural Rendering do the magic.

How it works

Under the hood it runs a dense Lucas-Kanade optical flow implemented in torch on CUDA - and it does it pairwise, one current-to-previous pair at a time, never moving the whole batch onto the GPU. That streaming habit is the whole pack's personality, and it's what lets this thing analyze long clips without melting a card. Analysis happens at a downscaled resolution (analysis_long_edge, default 512), because you're estimating motion, not refining detail - flow at 512 long-edge is plenty and far cheaper. Between each pair it also computes a scene-cut score, and when that score crosses scene_cut_threshold (default 0.22) it marks a cut. reset_on_scene_cut = true then zeroes the flow at that boundary, which is exactly what stops a temporal enhancer from smearing texture across a hard edit.

A few honest details from the source: engine = auto resolves to the built-in torch_lk path. The nvidia_optical_flow choice is reserved - it's not wired into this build, so selecting it will error rather than fall back. Just use auto.

The modes and inputs that matter

• motion_mode - compact_flow (default) stores FP16 downscaled flow vectors, the long-video answer. scene_cuts_only skips storing flow entirely and only records cuts - cheapest of all, and enough if the downstream work only cares about shot boundaries. full_flow stores flow at full source resolution; only for short clips or diagnostics.
• scene_cut_threshold - raise it if you're getting false cuts on fast camera moves; lower it if real edits are slipping through.
• quality - fast / balanced / quality trade flow-accuracy presets against runtime.
• storage_precision - FP16 by default; switch to float32 if flow math seems marginal.
• preview_frames - how many frames the flow_preview output visualizes (cap 32).

Three outputs: the motion packet (custom AETHERSCALE_MOTION type - wire it straight into the Neural Rendering node's motion input), flow_preview (a color-coded image of the motion vectors - or, in scene_cuts_only mode, a compact timeline bar with cuts marked as bright ticks, which is a lovely way to sanity-check your threshold), and stats JSON with the per-pair scene-cut scores and storage sizes.

Installing and running it

Manager → search AetherScale, or clone it in:

cd ComfyUI/custom_nodes
git clone https://github.com/vizart-vj/ComfyUI-AetherScale.git

Restart, and remember the environment contract shared by the whole pack: Windows, an RTX card, Python 3.10+, and internet on first use. Note this node itself doesn't need the NVIDIA runtime - it's pure torch/CUDA - but it exists to feed Neural Rendering, which does.

The trap to avoid

Motion Analysis is not a quality node. If you run it and stare at the flow preview waiting for an enhancement, you're looking at diagnostics. Its entire job is producing a motion packet that matches the resolution of the frames you'll later feed Neural Rendering - and that match matters: Neural Rendering will throw a hard error if the packet resolution doesn't line up with its input. So build the workflow as one graph: load frames → Motion Analysis → Neural Rendering, same frames in both, and don't resize in between.