cv2.ximgproc.covarianceEstimation

Some nodes are for everyone and this is not one of them. covarianceEstimation slides a window over an image and estimates the covariance of the data inside it at every position, which gives you a map of how the signal varies directionally - the anisotropy of local structure, the correlation between channels, the noise correlation you need if you're going to build a proper whitening filter or a Mahalanobis distance.

In a ComfyUI graph, that's a research tool. You reach for it when you're doing texture analysis, characterising a denoiser's noise, or quantifying local orientation. If you're generating pictures, you probably never open it. The reason it's in this article at all is that it's the kind of wrapper this pack exposes - and there are two traps in it that will waste an afternoon.

Trap one: the input must be complex

The tooltip says it outright: "The source image. Input image must be of a complex type." Not "should preferably be". A regular IMAGE reaching cv2 as uint8 BGR will simply not work. Complex data in OpenCV means a two-channel float array (CV_32FC2), which in this pack is what you get out of cv2.dft when you ask for the complex output (via CV DFT Flags), or from the quaternion family (cv2.ximgproc.createQuaternionImage). So the realistic chain starts with a DFT, which means thinking about the flags on the way in: DFT_COMPLEX_OUTPUT for a complex result, and CV DFT Flags exists precisely so you don't have to remember the bit values.

That also tells you what the node is for in practice: statistics of the frequency-domain representation, not of your photo.

Trap two: the window size is the statistics

• src - the complex input described above.
• windowRows, windowCols - the sliding window, in pixels. The tooltip explains the trade directly: "The window size parameters control the accuracy of the estimation. The sliding window moves over the entire image from the top-left corner to the bottom right corner. Each location of the window represents a sample. If the window is the size of the image, then this gives the exact covariance matrix. For all other cases, the sizes of the window will impact the number of samples."

Read that twice, because it's the whole node: window = whole image gives you one covariance estimate (the exact one), and anything smaller gives you a field of local estimates whose reliability depends on how many samples fit in the window. A 3×3 window over a 512×512 image produces 510×510 covariance estimates, each based on nine samples. That's a lot of noise wearing a lab coat. If you want the simple answer - "what is the covariance of this image" - set the window to the image size and take the single number.

Output is an NPARRAY. The pack ships no tooltip for the return, so before writing anything on top of it, run Inspect CV Data on it once to see the shape and dtype you're actually being handed. That's not a cop-out; it's the correct workflow for any auto-generated wrapper without doc text.

Where it connects

The natural consumer is cv2.Mahalanobis, which takes a covariance matrix and gives you a distance - the pack has it, and the pair makes a defensible "how unusual is this pixel patch" metric. Per-pixel covariance estimates can also be reduced to a scalar anisotropy measure and then fed through CV CV To Mask or CV Mask To CV to become a mask, which is how a research primitive finds its way into a real graph. And the pack's CV Array Statistic node handles the summary-statistics half if you don't need the full matrix machinery.

Installing it

Contrib module, part of ComfyUI CV. Manager → search ComfyUI CV, or:

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"
# restart ComfyUI

Python ≥3.12 and a recent, V3-API ComfyUI.

What goes wrong

• A cv2 assertion about type. Your input isn't complex. Route it through cv2.dft with DFT_COMPLEX_OUTPUT.
• Window bigger than the image, or zero - the defaults are all zeros, and zero-sized windows are not a "use something sensible" signal.
• An output that looks like noise. It is; you gave it a nine-sample window.
• ximgproc submodule empty. Contrib-only, and all four opencv-python* wheels share one site-packages/cv2, so a non-contrib install quietly leaves empty stub modules. tools/repair_opencv_contrib.py --check in the pack repo diagnoses it.
• Assuming it's a filter. It processes nothing and returns no image. It's a measurement, and the measurement is the point.