Motion Vector Extractor

Every H.264 clip you've ever touched already knows how every block of pixels moved between frames - the encoder computed it to compress the thing. This node just reads it back out and hands it to you as motion strokes. No optical flow network, no VRAM, no API. Point it at a video, get a JSON blob of point trajectories you can drop into a motion-brush workflow.

"Motion brush" is the idea where you paint strokes on a frame to tell the generator what should move and how - the same trick as Pika's "make this area move" or the vk-motion-brush ComfyUI workflows floating around. Hand-painting those strokes is fiddly. This node is the automation: it derives them from real footage's motion vectors, the ground truth of what actually moved.

How it works

The node is a thin ComfyUI wrapper around LukasBommes' mv-extractor, specifically its VideoCap class - a drop-in replacement for OpenCV's VideoCapture that decodes the inter-frame motion vectors baked into H.264 and MPEG-4 Part 2 streams, along with each frame's I/P/B type.

Here's the pipeline, from the node source:

1. Open video_url with VideoCap. If it can't open it, you get a RuntimeError: Could not open ....
2. Read up to frame_length frames, collecting the raw motion vectors.
3. Drop any vector whose |dx| + |dy| is ≤ scale - tiny drift becomes noise, so it's filtered.
4. Chain matching endpoints across frames into trajectories (a list of points per stroke), dividing coordinates by scale.
5. Return the whole thing as json.dumps - a list of polylines like [[[12, 40], [18, 52], [18, 64]], ...].

The output points are on a coarse grid relative to the source video, which is exactly the resolution a motion-brush condition wants. The whole thing runs on CPU in milliseconds. That's the appeal: it's practically free.

The inputs that matter

• video_url - a path to a real video file on disk. Not a URL, despite the name; it's a local path. And not just any file: it has to be H.264 or MPEG-4 encoded, because the node reads motion vectors from the compressed stream. That's why the pack's sample workflow re-encodes frames first - see the companion VideoCombineThenPath node.
• frame_length - how many frames of motion vectors to read before stopping. Default 2 is the bare minimum; the example workflow uses 14. More frames = longer, more useful trajectories, and a slower run.
• width / height - clamp the vector coordinates into the canvas. Defaults 320 × 576, matching the tall format the author's own tooling uses.
• scale - the noise filter and downsampling factor, default 4. Raise it to keep only strong motion; lower it to keep fine movement.

Installing it

The usual clone-and-restart, but with one extra step the README insists on:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI-Motion-Vector-Extractor
cd ComfyUI-Motion-Vector-Extractor
pip install motion-vector-extractor

Or just search "ComfyUI-Motion-Vector-Extractor" in ComfyUI Manager. Note there's no requirements.txt in the repo - the pip install motion-vector-extractor step is the dependency; skip it and the node dies at runtime with an ImportError. The package pulls in numpy and opencv-python itself.

Where people get burned

It's Linux x86-64 only. The motion-vector-extractor wheels are manylinux builds - there are no Windows or macOS wheels. The README says Python 3.8–3.10 and Ubuntu/Debian, and while the PyPI build now covers Python 3.9 through 3.14, the platform restriction hasn't changed. If you're on Windows, this pack is a non-starter unless you're willing to run ComfyUI in WSL. Check the platform before you burn an hour of your life.

Otherwise, the failure modes are simple: a wrong video_url path gives you "Could not open", a non-H.264 file gives you empty vectors, and there's no built-in viewer - the node just emits a string. Paste it into ShowText|pysssss to eyeball it, or load a frame plus the JSON into the pack's tools/draw.html "TRAJ MOTION DESIGNER" to draw the strokes over the image. That little HTML page is also where you hand-edit trajectories before feeding them onward.

One honest caveat: this is a niche, one-commit-ish pack from 2024, and the output is raw JSON with no standard consumer node. You'll be wiring the motionbrush string into whatever motion-brush pipeline you're using yourself. If that sounds like a project, this is the cheap way to get real motion data for it.