Sequence Mask

Batches of text are padded - you jam sequences of different lengths into one rectangular tensor, and the empty space is filled with dummy tokens. But you don't want your model learning from that dummy space. CdlSequenceMask is the ComfyDL node that erases it: positions past each sequence's valid length get overwritten with a mask value (usually 0), so padding stops contaminating attention, loss, and predictions.

It's the sequence_mask utility from the Dive into Deep Learning NLP chapters, and it's the quiet workhorse behind every masked operation in the pack - the RNN language models, the attention mechanisms, the seq2seq builders. If you're building any NLP-ish workflow in ComfyDL, this node shows up somewhere in the chain.

How it works

For a batch tensor of shape [batch, seq_len], plus a valid_len tensor holding each sequence's real length, it builds a boolean mask (position < valid_len) and overwrites everything at or beyond the valid length with mask_value. Simple, and it's the exact thing you want before:

• a masked softmax - you mask first, then softmax, so padding positions get probability ≈ 0 instead of sharing the weight
• a loss computation - you don't want the model rewarded or punished for predicting padding tokens
• feeding an RNN - so the network doesn't "learn" that sentence continuation is always a pad token

The inputs

• X - the [batch, seq_len] (or [batch, seq_len, ...]) sequence tensor.
• valid_len - a tensor of effective lengths, one per sequence. Rows shorter than their valid_len are untouched; rows that are all padding get fully masked.
• mask_value - what to write into masked positions (default 0.0, and that's the right default for most setups).

Output: masked, the same shape as X with padding positions rewritten.

Installing ComfyDL

The standard light install:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL
cd ComfyDL && pip install -r requirements.txt

Restart ComfyUI; it's under ComfyDL → TorchOps, or search "ComfyDL" in ComfyUI Manager. Only extra dependency is matplotlib.

The gotchas that actually bite

valid_len shape matters. If it's a scalar tensor it broadcasts to every row, which is fine; if it's a per-row vector it must have batch entries, matching the first dim of X. Get that wrong and either nothing masks or everything does - silently, in both cases, because a [batch, seq_len] boolean mask broadcast is forgiving in ways that hide bugs.

Second: mask then compute, in that order. Masking after a softmax doesn't zero out the padding's attention - the probability mass was already distributed. The pack's Masked Softmax node exists precisely because this ordering matters, and if you've wired things the wrong way around, your attention values will be subtly off and your loss will be quietly weird.

And remember the mask value should usually be 0 - but if you're masking for a loss that uses log-probs, you may want a very negative fill instead so masked positions contribute nothing after the softmax. The widget's range goes to ±1e9 for exactly that reason.