QFT Eigenvalue SR Saliency

This node answers a question almost nothing else in ComfyUI bothers to ask: where does your eye actually go in this image? It's a genuine visual saliency map - white where the image is visually "interesting," black where it's predictable background - computed with classical signal processing instead of a neural network. No model download, no VRAM, no prompt. It sits under image/analysis as "QFT Eigenvalue SR Saliency."

How it works

The classic Spectral Residual trick (Hou & Zhang, CVPR 2007) is: take the FFT of an image, subtract a smoothed version of the log spectrum, and the leftover "surprise" - the part of the spectrum that isn't predictable - is where saliency lives. Most color versions of this treat each RGB channel independently and smooth the log-amplitude with a box filter as the "prior."

This node replaces that box filter with something smarter. It computes a separate 2D FFT per color channel, then at every frequency builds a 3×3 Hermitian cross-spectral matrix by box-filtering each channel's spectrum against the conjugates of the others. That matrix captures how much the color channels are correlated at each frequency, not just how much energy is there. The node takes the largest eigenvalue (the default "dominant" mode), uses 1 / sqrt(λ) as a per-frequency weight, inverse-transforms each channel, and sums the squared magnitudes. A Gaussian blur and min-max normalize squash the result into a [0, 1] float image.

The upshot: frequencies where the color channels are strongly and predictably correlated get suppressed, and unusual cross-channel structure pops - which is why it finds salient regions the box-filter version misses. The README claims ~6x better center/border contrast on a synthetic test. All of it is batched numpy (np.linalg.eigh over the whole (H, W, 3, 3) array, no per-frequency Python loops), so it's correct, just not GPU-accelerated.

The inputs that matter

You set basically four things:

• window_size (default 15, 3–63) - the local frequency window. Bigger = smoother, closer to plain box-filter SR; smaller = more locally adaptive but noisier. The README's 9–21 range is a good starting point, and the code bumps even values up to the next odd one so you don't have to sweat that.
• gaussian_sigma (default 8, 0–100) - blur on the final map. Scale it with resolution; 8–16 is right for 512×512. Set it to 0 to skip the smoothing entirely (expect speckle).
• eig_mode - dominant (largest eigenvalue, safest default), mid, or minor (most sensitive to subtle cross-channel anomalies, also noisiest). One aside: the UI calls the default "dominant (λ₁)" but NumPy's eigh returns eigenvalues ascending, so the dominant one is index 2 under the hood. Same thing, different label.
• output_mode - grayscale or heatmap (a pure-NumPy jet colormap; no matplotlib dependency, which I appreciate).

One input, image, and it accepts batches. Feed it a single-channel image and it duplicates the luma into RGB rather than crashing - a thoughtful touch.

The single output, saliency_map, is an IMAGE at the same resolution as the input, float32 in [0, 1] - brighter = more salient. It is not a mask yet: it's a continuous map, so convert it to a mask (threshold, or feed through an image-to-mask node) before wiring it into inpainting or compositing.

Installing it

From ComfyUI Manager, search ComfyUI_EigenQFT_SRSM and install. Or, the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/bemoregt/ComfyUI_EigenQFT_SRSM

Then, optionally but recommended, pip install scipy - the node works fine without it, falling back to a pure-NumPy box/Gaussian filter implementation, but scipy's versions are faster and more standard. Only numpy and torch are hard requirements, and both ship with ComfyUI. No models to download, no CUDA pins, nothing heavy. Restart ComfyUI and you'll find it under image/analysis.

Troubleshooting

• It's slow on big images. Everything runs on CPU in float64, including a full eigen-decomposition per frequency. On a 1024×1024 input you'll notice the wait. Keep inputs reasonable or accept it as an offline analysis step - it's not something you'd run inside a generation loop.
• Very small window_size (3–5) gets noisy. That's documented behavior, not a bug. Raise it toward 15 if the map looks like static.
• Missing scipy doesn't break anything. It degrades to NumPy fallbacks, so if your install "failed" but the node loads, it didn't.
• The output won't composite cleanly as-is. Remember it's a [0, 1] float image, not a binary mask. Threshold it or convert it before chaining into InpaintModelConditioning or an image composite node, or you'll wonder why "the mask" has soft gray everywhere.

Saliency is a solved-enough problem that you won't reach for this often, and this is a single-node pack from a small author - expect "works as documented" energy rather than a big support community. But when you need to find the subject without asking a model - auto-generating a mask for region-weighted generation or attention-guided inpainting, or just curious why a render feels busy - it's the only node doing this job, and it does it with refreshingly few moving parts.