MotionVectors
Dense optical flow without the OpenCV homework
- images
- motion_vectors
- visualization
- stats
MotionVectors estimates the motion of every pixel between adjacent frames and hands it to you as an image batch. Two channels of flow per frame, plus an optional colour-coded preview so you can actually look at the field. If you're stabilising, retiming, flow-blending two takes, interpolating between frames, or building a matte that follows motion, this is the pass you'd normally write a script for.
It's in Radiance's VFX section because flow is a compositing primitive - the thing you need before the clever part. The KB's post-processing doc is right that ComfyUI's coverage of this layer is patchy and that people leave for a compositor more often than they should have to; this node is one of the small places that gap gets filled.
The solver choice is the whole story
preset is Fast, Medium or Ultra, and it means different things per solver. With DIS (the default) it's OpenCV's ultrafast, fast or medium preset - mostly a speed trade with similar accuracy. With Lucas-Kanade it's window radius 3, 5 or 9 px, which is a genuine accuracy/robustness trade.
solver is optional and defaults to Auto, which picks DIS and only falls back to Lucas-Kanade if OpenCV is missing. That default is correct and you should probably leave it. The author's own note is the reason: DIS holds a dense field out to about 20 px of motion and is roughly 11× faster. Lucas-Kanade was the previous solver - accurate to about 8 px, after which the field thins out even though the median stays close. That "median stays close" line is the trap: a sparse field can still look right in a preview while giving you nothing useful on fast motion. If your clip has anything moving faster than 8 px per frame and you forced Lucas-Kanade for some reason, that's your missing data.
flow_scale (default 1.0, range 0.1–10) multiplies the output vectors. At 1.0 the vectors are in pixel units at the input resolution; scale up when you want the field to read at a glance, scale down when you're feeding something that expects normalised motion. It scales the output, it doesn't change how the flow was estimated.
visualize off means you get the flow data and no diagnostic image. Turn it on and the colour-coded field shows up on the visualization output - the standard hue-is-direction, saturation-is-magnitude convention, and genuinely the fastest way to see whether a solve went wrong. Big flat regions with no hue change are a solver saying "I don't know". The third output, stats, is a JSON string with per-frame numbers, which is what you'd wire into a note or a log when you're processing a long clip unattended.
Install
Ships with Radiance. ComfyUI Manager → Radiance → install → restart → refresh, or:
cd ComfyUI/custom_nodes
git clone https://github.com/fxtd-studios/radiance.git
cd radiance
python -m pip install -r requirements.txt
The only dependency that matters is opencv-python, which is the first line of requirements.txt. OpenCV also supplies the EXR codec path when the OpenEXR wheel isn't available, so it's load-bearing in this pack twice over.
Where people get burned
- Getting the vector order wrong downstream. DIS returns x/y in the array's own layout, and half the "my flow is swapped" bug reports on the internet are really a transpose somewhere else. Check with
visualizebefore you blame the solver. - Feeding it 8-bit video and expecting clean flow. Flow wants detail. A heavily compressed clip gives you noisy vectors in flat areas - that's the codec, not the estimator.
- Long batches. Frames go through as float32 tensors, and a whole 4K clip's worth of flow is a lot of RAM. Radiance docs make the same point about Read's
max_video_frames; the same caution applies here. Process in chunks. - Assuming frame 0's output means something. There's no "previous" frame for the first frame, so the first entry in the outputs is inert. Drop it or handle it.
Inputs (5)
| Name | Type | Default | Description |
|---|---|---|---|
| images | IMAGE | Batch of frames to analyze. | |
| preset | COMBO | Medium | Solver effort. DIS: ultrafast, fast or medium preset (mostly a speed trade, similar accuracy). Lucas-Kanade: window radius 3, 5 or 9 px. |
| flow_scale | FLOAT | 1.00.1–10 | Scale factor for output vectors. 1.0 = pixel units. |
| visualize | BOOLEAN | false | Outputs a color-coded visualization of the motion field. |
| solveropt | COMBO | Auto | Auto uses DIS, and falls back to Lucas-Kanade only if OpenCV is missing. DIS holds a dense field out to about 20 px of motion and is roughly 11x faster. Lucas-Kanade is the previous solver: accurate to about 8 px, after which the field thins out even though the median stays close. |
Outputs (3)
| Name | Type | Description |
|---|---|---|
| motion_vectors | IMAGE | — |
| visualization | IMAGE | — |
| stats | STRING | — |