cv2.findTransformECC (1/2)

ECC - enhanced correlation coefficient - is the other way to align two images. No keypoints, no matching, no RANSAC. It optimises a warp so that one image's intensities correlate with the other's as strongly as possible. That makes it the right tool when your two frames are the same scene, slightly moved: frames of a burst, a golden reference template, a photo pair shot from a tripod, a document that got nudged between scans.

This node is the full-parameter overload of cv2.findTransformECC. Every knob cv2 has is here, and for once that's not a misleading boast: motionType, the termination criteria, the mask and the pre-blur size are all exposed rather than hidden.

The odd one out: warpMatrix is an input

Most estimation nodes return a matrix. This one takes one and returns it. warpMatrix is a required NPARRAY input - the initial guess - which cv2 refines in place, and the refined result comes back as the node's second output. With no better information you feed it an identity: for MOTION_TRANSLATION, MOTION_EUCLIDEAN and MOTION_AFFINE that's a 2×3 float matrix; for MOTION_HOMOGRAPHY it must be 3×3. (cv2 ignores the third row of a 3×3 when the motion type is one of the first three.)

This is also where the honest limitation lives: ECC is a local optimiser. If the frames are a couple of degrees and a handful of pixels apart, identity is a fine starting point. If they're badly displaced or rotated, you need a rough alignment first - a similarity transform from a cheaper estimator, or the previous frame's result in a tracking loop - or the optimiser marches off toward a wrong local maximum and reports it with confidence.

Inputs that matter

templateImage and inputImage are the two frames: 1- or 3-channel, CV_8U/16U/32F/64F, mutually the same. The sockets are image-ish, so IMAGE, MASK or NPARRAY all work - an IMAGE arrives as uint8 BGR frame 0, which is the normal case for a ComfyUI graph.

motionType sets the model: MOTION_TRANSLATION (2 degrees of freedom), MOTION_EUCLIDEAN (shift plus rotation, 3), MOTION_AFFINE (6, the default) or MOTION_HOMOGRAPHY (8). Fewer degrees of freedom converge more reliably, and the failure mode of too many is a warp that fits noise. If your images differ only by a shift, don't ask for a homography.

criteria_type / criteria_max_count / criteria_epsilon are one cv2 TermCriteria split into three widgets - stop after n iterations, stop when the improvement drops below epsilon, or whichever comes first. The defaults are 30 iterations and 0.001, which are conservative: a real alignment usually wants more iterations and a tighter epsilon.

inputMask is a single-channel mask saying which pixels of inputImage are valid. Useful for letterboxed frames, watermarks and sensor artefacts.

gaussFiltSize deserves a warning. The widget defaults to 0, but OpenCV documents 5 as the default and this parameter wants an odd size - it's a pre-blur that stops the optimiser chasing sensor noise. Set 5, or another odd number, rather than trusting the default. The curated CV Find Transform (ECC) node forces an odd value for you, which is one of several reasons it's often the better pick.

Outputs

float is the final ECC correlation coefficient - cv2's return value, 1 meaning a perfect match. It's your quality score, and it's the number to log and branch on. nparray is the refined warp matrix, ready for cv2_warpAffine. Apply it with WARP_INVERSE_MAP in the flags and dsize set to the template's size, which is what pulls the input into the template's frame - the shipped exercise_image_registration_ecc.json does exactly that, and shows the payoff (aligned stack average, sharp; unaligned, ghosted).

The failure mode to know about

OpenCV's own documentation is explicit: the function throws when the algorithm doesn't converge - featureless frames, unrelated images, degenerate contrast. In a ComfyUI graph a cv2 exception stops the run. If you're processing a folder where some frames are junk, the curated CV Find Transform (ECC) node exists precisely because of this: it catches the non-convergence and returns found = false with an identity matrix, so a control-flow node can skip the bad frames instead of killing the queue.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

Manager users: "ComfyUI CV", install, restart. Python ≥ 3.12 and a V3-API ComfyUI.

Common issues

OpenCV error, workflow dead. Non-convergence, as above. Shrink the problem: grayscale the frames, downscale before aligning and scale the resulting matrix back up, or start from a rough warp instead of identity.

It converges to a wrong answer. ECC is iterative and greedy; with big displacements it lands in a local optimum. Give it a starting matrix that's roughly right.

Deceptively slow runs. Whole-frame alignment on 4K pairs with 500 iterations is not fast, and it's CPU work. Resize first, refine second.