CLIP Score

Every other metric in this pack asks "is this image good?". CLIP Score asks a different question: "does this image match the words?" A tack-sharp photo and a blurry mess score the same if both of them show the thing the caption describes. If you're testing prompt adherence - does my new LoRA actually follow the outfit description, does this checkpoint ignore the subject when I push the style - that's the metric you want, and it's the one no generic score node can offer, because it needs a second input nobody else asks for: text.

Why it's its own node

pyiqa's clipscore is OpenAI CLIP ViT-B/32 with the paper's 2.5× scaling, giving a range of roughly 0–2.5, higher better. The weights come from (Down)load pyiqa Model - architecture CLIP Score - like every other metric in the pack. What can't come from there is the caption, so the node takes it as a widget.

The inputs

model from the loader (architecture CLIP Score). image, the batch to score. caption, a multiline text box, and this is where the node is unusually opinionated.

The rules, straight from the author's tooltip: one line applies to every image in the batch; one line per image pairs them up in order. Any other count is refused, with both numbers in the error. Blank lines are ignored, because a trailing newline in a text box isn't a caption.

That strictness is deliberate and it's the right call. Silently rounding "3 captions, 5 images" down to the first three would give you a plausible-looking number that means nothing. The node stops instead. So the normal case is: generate eight images from one prompt, paste that one prompt in, read eight scores.

memory_management defaults to unloading the metric after scoring. Leave it unless you're scoring repeatedly and want to skip the reload.

Output is a score socket, one value per image.

How to read the number

Higher is better and the scale tops out around 2.5. In practice everything lands in a narrow band, so treat it as a comparator, not a grade: run the same prompt through four LoRAs, sort, and the ranking is meaningful in a way that the absolute value is not. A gap of 0.02 between two runs is noise, not a finding.

Second calibration point: CLIP ViT-B/32 was trained on alt text and it's a coarse instrument. It's reasonably good at "is there a dog in this picture" and noticeably bad at word order, negation and counting. "A cat sitting on a mat, photorealistic" and "a mat sitting on a cat, photorealistic" look similar to it. So it tells you the subject made it in, not that the composition is right.

And it's alignment, not quality - this is the single most common misreading. If you want quality, that's pyiqa NR Score or FGResQ NR Score. Use both if you've got a prompt A/B going: alignment tells you which prompt stuck, an NR metric tells you which one produced a cleaner image, and yes, they frequently disagree.

The practical workflow

Generate the same seed with four prompt variants → batch them together → one caption per image in the text box, matching the wire order → five scores out. Pipe the scores into a text display or into the filename of a Save Image node and you've got a table. That's the useful part: it makes an aesthetic argument into something you can rerun tomorrow.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/AnsteinHuynh/ComfyUI-pyiqa-toolbox
python -m pip install -r ComfyUI-pyiqa-toolbox/requirements.txt

Or ComfyUI Manager → search pyiqa-toolbox. Restart. The ViT-B/32 weights (~350 MB) download on first use into ComfyUI/models/pyiqa.

Worth knowing before you install: pyiqa resolves transformers>=5, and in a shared ComfyUI environment that can break older custom nodes pinned to 4.x. Standard ecosystem hazard - same story as every pack that pulls a big library - but this one has a named culprit, so don't be surprised.

Common issues

"CLIP Score got N captions for M image(s)". Exactly what it says: you have a mismatch. Use one line for the whole batch, or exactly one line per image.

"No module named clip". pyiqa gets clip from its own openai-clip dependency, so a normal install has it. A trimmed environment might not:

python -m pip install openai-clip

Empty caption. Refused, and the message reminds you it measures image-to-text match, not quality. That's the node being helpful rather than pedantic.

Every image scores nearly identically. With one caption broadcast over a batch from the same prompt, they will. Vary the prompt, not the seed, if you want spread.