FGResQ FR Score

Most full-reference metrics give you a distance: two images in, one float out, and you're left interpreting whether 0.11 is better than 0.09. FGResQ's pair mode does something more useful. You hand it two images and it hands back a quality score for each one plus a verdict: which of the two is better, or that they're basically the same. That's the compare slider, converted into a string you can branch on.

What comes back

Five outputs, and the first three are the whole point:

target_score and reference_score are FGResQ quality in 0–1 for each side. Both, from one call - so you get the pairwise judgement and the individual grades in the same pass.

rank is the pairwise verdict as pyiqa's integer code: 0 = reference is better, 1 = target is better, 2 = similar.

verdict is the same thing in words - target_better, reference_better, similar - which is what you want wired to a display or a filename, because nobody reading a log wants to look up what 0 meant.

rank_prob is the softmax probability of the class it picked. Treat it as a confidence dial. Three classes means a uniform guess sits around 0.33, so a verdict at 0.36 is the model shrugging at you; 0.9 is it being fairly sure. If you're auto-filtering a batch on this, gate on rank_prob as well as rank, or you'll throw away images on coin flips.

Order matters here, unlike everywhere else

Every other FR metric in this pack is order-independent, mostly. FGResQ's pair mode is emphatically not: swap target and reference and you swap the two scores and can flip the rank. The tooltips say so twice, which is a hint about how often people got it wrong.

The convention is consistent: target is the image being judged - the restored, denoised, upscaled, newly generated one. reference is the pristine original or the competitor. So verdict = target_better reads as "the thing I made beat the thing I compared it to". Learn the pyiqa quirk in the tooltip too - pyiqa names its classes image1/image2 because it sees arguments in a fixed order, and the node maps x0 to target and x1 to reference so the codes come out sensible.

Pairs are matched by batch position: target #1 against reference #1, and so on. Same batch size on both sides.

Where this is the right tool

Upscaler bake-offs. Feed the same source through two pipelines, wire the two outputs to reference and target respectively, and you get a per-image winner plus two grades - no eyeballing, no slider, and you can run it over 200 images and count wins. Same for denoise strength sweeps, LoRA step comparisons, or "did my img2img pass actually improve this or just smooth it?"

It's also the honest answer for the aesthetic comparison case. If both images are plausible but different, LPIPS will mostly measure content mismatch and tell you nothing useful; a pairwise quality judgement is the right shape of question for that.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/AnsteinHuynh/ComfyUI-pyiqa-toolbox
python -m pip install -r ComfyUI-pyiqa-toolbox/requirements.txt

Or ComfyUI Manager → search pyiqa-toolbox. Restart; the node is under pyiqa/FGResQ. There's a shipped example graph - Workflow → Browse Templates → ComfyUI-pyiqa-toolbox, the FR one compares against LPIPS+ with FGResQ FR doing the ranking.

Weights auto-download on first score into ComfyUI/models/pyiqa. A shared models/pyiqa/FGResQ/ folder is where custom FGResQ fine-tunes go, and they show up as extra entries on the loader's variant list.

One warning that isn't optional: pyiqa pulls transformers>=5 on install, and that upgrade can break other custom nodes pinned to 4.x. Better to know now than at 2am.

Common issues

Verdict says similar for two images that clearly differ. That's what the metric thinks; check rank_prob. Low confidence plus "similar" is the model declining to call it, not a bug.

Verdicts that look like plain CLIP judgements. The published FGResQ checkpoints carry a vision_model. key prefix that transformers ≥ ~5.6 dropped; pyiqa only warns, so the fine-tuned backbone silently doesn't load and you're scoring with stock HF CLIP. Current versions of the pack strip the prefix. Update the pack before you trust a suspicious ranking.

Batch-size mismatch. target and reference must match. One reference for a batch of eight is an error.

Two score nodes, one loader, sudden reloads. memory_management unloads after scoring by default; if you're feeding a pair node and another score node from one loader, set keep loaded on all but the last.