BLEU Score

CdlBleu computes a BLEU score between a predicted sentence and a reference sentence - no tensors required. BLEU is the standard metric for machine translation and text generation: it compares n-grams (runs of n words) in your output against the reference and combines how many match with a penalty for outputs that are too short. This node is a direct port of the simplified BLEU from Dive into Deep Learning, and it's one of the few ComfyDL nodes you can use completely standalone, since its inputs are plain text.

How it works

You type or paste two sentences, it splits them on spaces, and computes the d2l version of BLEU. Two parts:

1. Brevity penalty. If the prediction is shorter than the reference, the score gets multiplied down - exp(1 - len_ref/len_pred) - so a model that emits one perfect word and stops doesn't game the metric. Overlong outputs aren't penalized by this term.
2. N-gram precision. For each n from 1 up to max_n, it counts how many n-grams from the prediction also appear in the reference (with clipping so repeated words don't count forever), divides by the prediction's n-gram count, and raises that to the power 0.5^n. The shorter the n-gram, the more it's weighted - matching a single word matters more than matching a 4-gram run.

The final score multiplies both together. Scores run 0 to 1, and 1.0 means "indistinguishable from the reference at every n-gram length you checked."

Inputs and output

• pred_seq - the generated text, default "the quick brown".
• label_seq - the reference text, default "the quick brown fox".
• max_n - how many n-gram lengths to check, 1 to 4, default 4.

The single output is bleu_score, a FLOAT. The defaults are chosen to be instructive: "the quick brown" against "the quick brown fox" scores below 1.0 because of the brevity penalty - a nice way to see the mechanism in action before you feed it real data.

Where you'd use it

This is the teaching-grade version of BLEU. In the ComfyDL world it pairs with the Seq2Seq / machine-translation example workflows - generate a translation, score it against the reference, watch the number. For anything production, the honest note: real BLEU implementations (sacrebleu, NLTK) handle tokenization, multiple references, and smoothing differently, and the d2l version is deliberately simplified. Use this node to understand BLEU, not to publish a benchmark number.

Installing it

It's part of ComfyDL, one install for the whole pack:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL
pip install -r ./ComfyDL/requirements.txt

Restart ComfyUI, or search "ComfyDL" in ComfyUI Manager.

Gotchas

• Tokens are split on single spaces. Punctuation counts as part of a token, and double spaces create empty tokens that skew n-gram counts. Clean your input: "the quick brown", not "the, quick, brown" or "the quick".
• Lower max_n loosens the score (fewer n-gram lengths to fail); the default 4 is the standard setting.
• The brevity penalty means a shorter-than-reference output can score surprisingly low even when every word it did produce is correct - that's BLEU being BLEU, not a bug.