ComfyUI Node

ImageScore

The scorer that crashes on its first run — here's why and the fix

By azure-dragon-ai·Created 3 years ago·Updated 2 years ago· 4
ImageScore
  • model
  • real_features
  • fake_features
  • SCORES
  • SCORES1
device

You found the ImageScore node (HaojihuiClipScoreImageScore) because you want a number out of this pack - the CLIP similarity between two images. Fair goal. So here's the honest headline: as shipped, this node crashes every time you run it. It's the one node in ComfyUI-ClipScore-Nodes that actually computes a score, and the code has a bug that makes it throw a NameError the instant it executes.

Let me show you exactly where. The scorer's core math is fine - normalize both feature vectors, take the dot product, scale by CLIP's learned logit_scale:

real_features  = real_features  / real_features.norm(dim=1, keepdim=True).to(torch.float32)
fake_features  = fake_features  / fake_features.norm(dim=1, keepdim=True).to(torch.float32)
score = logit_scale * (fake_features * real_features).sum()

That gives you a scaled cosine similarity - higher means the fake is closer to the real in CLIP space. Then the author tried to average scores across runs:

score_acc += score
sample_num += 1
scores = score_acc / sample_num

...and score_acc and sample_num are never defined anywhere in the file. No globals, no initialization. So the first run dies with NameError: name 'score_acc' is not defined. The intended behavior was a running average accumulated across batches; the author wrote the accumulator lines and forgot to declare the variables.

The fix

You have options, from quick to proper. Quickest: make it return the single score instead of the broken average:

score = logit_scale * (fake_features * real_features).sum()
scores_str = str(score)
return (scores_str, score)

Or, if you actually want the running-average behavior across a batch (accumulate features for several images, then average), initialize the accumulators as module-level globals near the top of clipscore.py:

score_acc = 0
sample_num = 0

Either way, edit the file in ComfyUI/custom_nodes/ComfyUI-ClipScore-Nodes/clipscore.py and restart ComfyUI. This is a single-commit, unmaintained pack from January 2024 - don't wait for a fix upstream.

Inputs and outputs

  • model - from the pack's Loader (PS_MODEL).
  • real_features - the REAL_FEATURES output of the Real Image Processor (your reference image).
  • fake_features - the FAKE_FEATURES output of the Fake Image Processor (your generation).
  • device - cuda or cpu.

Two outputs:

  • SCORES (STRING) - the score formatted as text, for a ShowText node or the pack's Save Text File pattern.
  • SCORES1 (FLOAT) - the same score as a number, if you want to route it into anything numeric.

Setting expectations

Two things to understand before you get excited about the numbers. First, the score is image-to-image similarity, not quality - it measures how close two images are in CLIP space, and CLIP is a fairly coarse "what's in the picture" model. Second, the processors feed this node only the first image of any batch (images[0]), so you can't batch-score without looping one image at a time. Combined with the crash bug, this pack is a toy, not a benchmarking rig.

Installing

Same pack, same story: ComfyUI Manager (search "ComfyUI-ClipScore-Nodes") or clone and restart:

cd ComfyUI/custom_nodes
git clone https://github.com/azure-dragon-ai/ComfyUI-ClipScore-Nodes

Then install the undeclared dependency:

pip install git+https://github.com/openai/CLIP.git

Run the Loader once before anything (it downloads CLIP weights from OpenAI), and remember the patch above - otherwise ImageScore is a guaranteed red error node from the first run onward.

CategoryHaojihui/ClipScore

Inputs (4)

NameTypeDefaultDescription
modelPS_MODEL
real_featuresREAL_FEATURES
fake_featuresFAKE_FEATURES
deviceCOMBO2 options: cuda, cpu

Outputs (2)

NameTypeDescription
SCORESSTRING
SCORES1FLOAT