HSWQ Quality Benchmark (Auto-Align)

You just converted your SDXL UNet to fp8 and the two images on screen look… basically the same? Good instinct, but eyeballing a before/after is a lousy way to find out whether the conversion quietly ate your fine details. This node is the honest answer: it takes a reference image and a target image and returns four real metrics - LPIPS, SSIM, CLIP similarity, and MSE - plus a diff image that shows you exactly where they disagree.

It ships in the HSWQ pack (the unofficial ComfyUI port of Hybrid Sensitivity Weighted Quantization), and honestly it's the most broadly useful node in that repo. You'll use it to verify an HSWQ fp8 conversion didn't wreck anything, but it's just as happy benchmarking any two images: quantized vs original, two samplers, two checkpoints. Nothing about it is HSWQ-specific.

How it works

Comparing fp16 and fp8 output is meaner than it sounds, because the images are never pixel-aligned - a tiny shift from slightly different memory layouts makes naive metrics scream "broken." So the node first runs its auto-align: a brute-force grid search over shifts up to ±12 pixels, comparing center crops, then cropping both images to their intersection. Then it measures:

• LPIPS (perceptual/texture, lower is better) - loads a small AlexNet-based LPIPS model and compares deep features, which is much closer to "does it look different" than raw pixels.
• SSIM (structural, higher is better) - the classic luminance/structure score, computed in-file.
• CLIP (semantic, higher is better) - open_clip's ViT-B-32 with OpenAI weights, comparing how similarly the two images embed. Two completely different images can still score ~0.7 here, so treat it as "did the meaning survive," not "are they identical."
• MSE (pixel, lower is better) - the raw diff that feeds the visualization.

Those get rolled into a letter grade (S through D) with sane thresholds, and the whole report comes back as a string plus individual floats.

Inputs and outputs that matter

You feed it two images and mostly leave the rest alone: image_ref is your reference (the original fp16 output), image_target is the thing under test (the fp8 output). compute_lpips and compute_clip default to on - turn them off if you want a faster run and don't care about those axes. enable_auto_align defaults to on and you should keep it that way. diff_amplification (1–100) multiplies the diff visualization: at 1 the diff is often too faint to read, at 20+ it turns into an obvious hot-spot map.

The diff_image output is that amplified, clamped absolute difference - wire it to a preview node. report_text is the full formatted report; the four floats (lpips_dist, ssim_score, clip_similarity, mse_error) are there so you can log or compare them programmatically. A clean HSWQ conversion at keep_ratio 0.25 typically lands in the A–B band; if you're seeing a C or D, your quantization is genuinely hurting the image and you should raise the keep ratio.

Installing it

It's part of the HSWQ pack, so install that whole repo once (ComfyUI Manager → search "HSWQ", or cd ComfyUI/custom_nodes && git clone https://github.com/Shiba-2-shiba/ComfyUI-Unofficial-HSWQ-QuantizerHSWQ-Quantizer, then restart). The benchmark's two extra dependencies are the part people forget:

pip install lpips open_clip_torch

Gotchas

The first LPIPS/CLIP run downloads pretrained weights, so it needs internet and can hang for a minute - that's normal. If the libs aren't installed, the node doesn't crash; it just reports 0.0 for LPIPS and 1.0 for CLIP, which will silently flatter you. Don't trust the grade until you've confirmed those packages are actually in your environment. And remember the score is a snapshot of one prompt - a good grade on one image doesn't prove your whole model is fine, it just proves this conversion didn't obviously break.