CV Kalman Filter Step

A Kalman filter does one thing beautifully: given noisy measurements of a moving thing, it estimates where the thing is and, more importantly, where it will be. The second half is what makes it indispensable for tracking, because the moment your detector loses the object - occlusion, motion blur, a frame where the face turns away - the filter keeps predicting instead of the track jumping to whatever the detector hallucinated that frame. cv2.KalmanFilter is a stateful class, which is precisely what the pack's ~470 auto-generated function wrappers cannot expose. CV Kalman Filter Step packages a whole filter state into one NPARRAY so you can carry it across frames on a wire.

The mechanism, and why it's shaped like this

The state - the posterior state vector plus its error covariance - is packed into a single float32 array of shape (n, n+1): column 0 is the state vector, the rest is the covariance. You're not meant to read it; you're meant to pass it along. Chain one node per frame: this node's state output into the next frame's state input. Leave the first frame's state unconnected and the filter seeds itself from the first measurement with zero velocity.

model picks the motion model:

• constant velocity (x, y + vx, vy) - the default and the standard tracker. It coasts in a straight line when the measurement drops out.
• constant position (x, y) - smooths jitter and nothing else. velocity comes back zero.
• constant acceleration (x, y + vx, vy + ax, ay) - follows curved motion, but needs a cleaner signal to be worth it.

dt is the time between steps, default 1.0, i.e. "pixels per frame". Bump it if your frames aren't equally spaced and you want velocity in real units.

The two noise knobs are the actual tuning, and they trade off in opposite directions. process_noise (default 0.01) is how much the model is allowed to be wrong per step: raise it to follow abrupt manoeuvres, lower it for a smoother, laggier track. measurement_noise (default 1.0) is how noisy your detector is in squared pixels: raise it to trust the model more than the measurement (more smoothing), lower it to snap onto every detection. If your track lags behind a fast object, that's process noise too low. If it jitters, it's measurement noise too low. initial_uncertainty only applies when there's no incoming state, and it's how much the seed position is doubted.

The inputs and outputs that matter

measurement is an optional NPARRAY: 2 numbers (x, y) as a 1×2 array, a CV Scalar literal, or any N×2 point set - the first row is used. Leave it unconnected, or pass an empty array, and you get a predict-only step: the filter extrapolates and the track coasts through the occlusion.

Outputs: state (chain it onward), filtered (1×2 position after correction - the smoothed track point, equal to predicted on a coasting step), predicted (1×2 where the motion model expected it before seeing the measurement - the extrapolation that carries you through the dropout), velocity (1×2, units per dt), and corrected (BOOLEAN: true when a measurement was applied, false on a coasting step).

Wire filtered or predicted into CV Draw Points, and draw velocity with CV Draw Rays if you want to visualise where the track is heading. Feed the measurement input from anything that produces a point: a CV Track Window centre, a Select Component At Point centroid, a YuNet face box. Choose predicted vs filtered deliberately when the detector is unreliable - that's the whole reason both exist.

Install

ComfyUI Manager, search the pack title comfyui_cv (bmad4ever/comfyui_cv). Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv

Restart ComfyUI. It needs Python ≥ 3.12 and a recent ComfyUI on the V3 node API - the pack is all io.ComfyNode/io.Schema with no NODE_CLASS_MAPPINGS, so an old install loads nothing. Dependency:

pip install "opencv-contrib-python-headless~=5.0.0.93"

Stay on the contrib wheel: all four OpenCV distributions share one site-packages/cv2, and a non-contrib install over a contrib one silently removes the contrib submodules, making contrib nodes vanish from the menu with no log message. tools/repair_opencv_contrib.py --check / --apply fixes it.

The real-world caveats

The graph shape is the hard part. You need one node per frame, and in a batch workflow that means a fold, not a plain chain - a foreach-style accumulator keeps one value per step, exactly what the state is. People try to do this with a single node and a loop and end up fighting it, so plan the graph before wiring.

Detector noise ruins tuning. Tune measurement_noise by watching your own detections bounce, not by copying a number from a tutorial. There's no universal right answer, and the tooltips tell you the direction to move.

A coasting track is a guess. During a long occlusion, predicted drifts. That's the deal you accepted; if the object reverses, the coast goes the wrong way. Use corrected to gate anything that must not act on a guess.

The pack's README is worth reading before trusting any of this in a pipeline: it's a personal project with heavy LLM assistance, several sample workflows tuned to specific data, no support promises, and its own warning against production use without independent review. A Kalman filter is textbook mathematics, but the packing convention (state as an (n, n+1) array you never read) is this pack's invention - so check it if you build on it.