cv2.findTransformECCWithMask

The plain ECC nodes give you one mask, and it applies to the image being aligned. This one - an OpenCV 5 addition - takes a mask for each image: a templateMask saying which pixels of the reference are trustworthy, and an inputMask saying which pixels of the target are. That's the difference between "align these two photos" and "align these two photos, ignoring the letterbox bars, the timestamp overlay, the tourist walking through frame two, and the ROI you don't care about".

ECC is intensity-based registration: it optimises a warp so the two images correlate as strongly as possible. That makes it excellent on near-identical content and vulnerable to anything in either frame that doesn't belong - a moving object drags the solution toward itself. Masking is how you tell it what to look at, and this entry point lets you be specific about both sides.

Inputs

templateImage and inputImage are the reference and the frame to be aligned. All the image-ish sockets here take IMAGE, MASK or NPARRAY; an IMAGE is converted to uint8 BGR frame 0.

templateMask and inputMask are both required, both single-channel 8-bit, and both must be the same size as their image. cv2 doesn't resize them for you, and a size mismatch raises. Non-zero pixels are the valid ones - so a mask built from a threshold, a segmentation, or CV Bounding Boxes to Masks drops straight in.

warpMatrix is required and is an input rather than something you type: cv2 refines the matrix you pass in place and the node returns the result. Feed an identity - 2×3 for the affine/euclidean/translation models, 3×3 for MOTION_HOMOGRAPHY. Remember ECC is a local optimiser, so identity assumes the frames aren't far apart; a badly displaced pair needs a rough prior or it converges to the wrong answer.

The optional controls are motionType (MOTION_AFFINE by default; smaller models converge more reliably), the criteria trio criteria_type / criteria_max_count / criteria_epsilon (30 iterations and 0.001 by default - conservative; raise both for real work), and gaussFiltSize, which presets to 5 here and smooths the images and the masks before alignment. That last detail is why feathered mask edges behave nicely: the same blur that suppresses sensor noise stops a hard mask boundary from acting like a fake edge.

Outputs

float is the final correlation coefficient - 1 is perfect - and nparray is the refined warp. Apply the warp with cv2_warpAffine (or cv2_warpPerspective for homography) using INTER_LINEAR | WARP_INVERSE_MAP and dsize set to the template's size. Same pattern as the other ECC nodes, and exercise_image_registration_ecc.json is the reference graph.

When to use it instead of the plain ECC node

Use it whenever the two frames don't have identical valid area. A reference plate with a clean region and a target with a different clean region. Footage with burnt-in text on one side. Scans where the paper edge moved. A stereo pair that isn't quite rectified, where the valid band overlaps only in the middle.

If both your frames are fully valid everywhere, this node buys you nothing over cv2.findTransformECC (2/2) - same optimiser, same outputs, more wiring.

And as with the other ECC wrappers, remember what the raw function does on failure: cv2 throws when the iteration doesn't converge, and in ComfyUI that stops the run rather than producing a bad matrix. That's fine for a batch where everything is known-good, unreasonable for arbitrary user input. The curated CV Find Transform (ECC) node is the failure-tolerant version.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/bmad4ever/comfyui_cv
pip install "opencv-contrib-python-headless~=5.0.0.93"

ComfyUI Manager: search "ComfyUI CV", install, restart. Python ≥ 3.12 and a ComfyUI built on the V3 node API - this node also needs an OpenCV recent enough to have the function, which is why the pack pins that contrib wheel version rather than floating it.

Common issues

cv2 error on mask size. Masks must match their image dimensions exactly. Resize with CV Array → Mask/cv2_resize in the same frame before wiring them, not after.

Masked input aligns worse than unmasked. Check the polarity of your masks: non-zero means valid, so an inverted mask is telling the optimiser that the only usable pixels are the ones you wanted excluded.

It won't converge on a legitimately small overlap. Shrink the problem - crop to the overlap, align, then apply the transform to the full frame. Iterative whole-frame alignment on a 4K pair is slow regardless, and CPU work.

"(-213) needs to be compiled" or missing contrib nodes. Some OpenCV functions are build-gated, and a non-contrib wheel installed over the contrib one empties the contrib submodules. The pack's tools/repair_opencv_contrib.py --check / --apply reports and repairs that state.