cv2.calcCovarMatrix

Covariance is the "how do these variables move together" matrix, and it's the input to a surprising number of classic CV routines: Mahalanobis distance, PCA, colour-statistics matching, any kind of "is this frame typical of the set" scoring. cv2.calcCovarMatrix computes it from a stack of vectors - one vector per row or per column, depending on flags you have to supply yourself.

It's a raw wrapper in comfyui_cv, the pack of ~470 cv2.* nodes. This is one of the abstract-maths corners of OpenCV rather than a picture filter, so it arrives as NPARRAY in, NPARRAY out, with no image semantics at all.

How it works

samples is one big matrix holding your vectors. flags is the whole game - it's a CovarFlags bitmask, and the node exposes it as a plain integer:

• COVAR_NORMAL (1) vs the default COVAR_SCRAMBLED (0) - normal gives you the actual covariance matrix; scrambled gives OpenCV's scaled/transposed variant, which is what you want only if you're feeding an eigen-solver.
• COVAR_USE_AVG (2) - use a mean you already have, instead of computing it.
• COVAR_SCALE (4) - divide by the number of vectors, turning sums of products into averages.
• COVAR_ROWS (8) / COVAR_COLS (16) - say whether your vectors are rows or columns. Without one of these, cv2 assumes scrambled mode.

So the value most people actually want is 9: COVAR_NORMAL | COVAR_ROWS. The default of 0 gives you scrambled output, which is a perfectly valid matrix and not the one you asked for. It's the same shape of trap as leaving normalize off a box filter - nothing errors, the numbers are just quietly different.

The ctype combo picks the working type and defaults to CV_64F, which is what you want; float64 keeps a covariance matrix from losing precision when you're averaging thousands of samples, and there is no reason to narrow it.

Inputs and outputs

• samples (required, NPARRAY) - the vectors, one per row or column per your flags.
• mean (required, NPARRAY) - the average. It's an in/out parameter in OpenCV: with COVAR_USE_AVG you pass your own, otherwise OpenCV writes the computed one here. The socket is required either way, which means you must feed it something even when you don't have a mean - a zero vector built with CV Scalar (e.g. (0, 0, 0) for three channels) is the usual throwaway.
• flags (required, INT, default 0) - the bitmask above.
• ctype (optional, combo, default CV_64F) - the intermediate/output type.

Two outputs, both NPARRAY: covar (the matrix) and mean (the average, whether you supplied it or it was computed). Neither is an image - to look at covar you'd send it to CV Chart Matrix for a labelled heatmap, or Preview CV Array, or CV Array To Text if you want to actually read numbers.

Where it earns a place in a graph

The honest answer is: analysis, not generation. Two uses keep coming up. First, per-frame colour vectors: reduce each frame of a batch to a small vector (mean per channel, or a histogram via the pack's CV Histogram), stack them, and the covariance tells you how much the set varies - then cv2.Mahalanobis with the inverse covariance scores how far any single frame sits from the group. That's a deterministic "which of these 200 outputs is the odd one out" filter, and it costs nothing next to eyeballing them.

Second, feeding PCA for a linear fit or a dimensionality reduction over a dataset of your own making. Both are cases where you'd otherwise be in numpy; the argument for doing it in-graph is that the numbers never leave ComfyUI.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv

Restart, or Manager → search "comfyui_cv". Python ≥ 3.12, recent ComfyUI on the V3 node API, and:

pip install "opencv-contrib-python-headless~=5.0.0.93"

No models; nothing to download.

Common issues

The matrix looks transposed or oddly scaled. You left flags at 0, so OpenCV produced scrambled output. Use 9.

"The number of rows/columns doesn't match" assertion. The mean vector length has to equal the number of variables (columns, if your vectors are rows), not the number of samples. A 500×3 sample stack wants a 3-element mean.

Garbage values. Watch the input dtype. A samples array that arrived from an IMAGE conversion is uint8, and covariance over 8-bit data with products of 255×255 in an 8-bit container isn't going to be pretty. Get to float first - ctype handles the output, not the input.