Gradient Clip

CdlGradClipping is the guardrail for training runs where gradients spiral out of control. In deep networks - especially RNNs and sequence models - a bad batch can push gradient values up so steeply that the next optimizer step flings your weights somewhere catastrophic, and the loss goes from "improving" to "NaN" in one step. Gradient clipping caps that by scaling the gradients down to a maximum norm before the update. This node is d2l's grad_clipping as a graph node: pick a threshold, and it rescues your training loop from the classic blow-up.

How it works

The mechanism is the global-norm clip, the standard d2l/PyTorch approach:

1. It collects the model's parameters that have gradients computed (requires_grad and a non-None grad).
2. It computes the global gradient norm - the square root of the sum of squared gradients across all of those parameters.
3. If that norm exceeds your theta, it scales every gradient by theta / norm, which pulls the total norm back down to exactly theta without changing any gradient's direction - the update still points the same way, just shorter.

That direction preservation is why clipping beats just lowering the learning rate: you keep the useful signal, you just refuse to take a jump so big it breaks the weights. The node also hands back the norm it measured, so you can log it and watch how close to the cliff you're running.

The critical prerequisite, spelled out in the code comments: gradients must already be computed. This node clips existing grads; it doesn't run backprop. It sits between your loss/backward step and the optimizer update (CdlSgdStep), not before them.

Inputs and output

• theta - required FLOAT, the max allowed gradient norm, default 1.0. Lower = more aggressive capping; 0.1–1.0 is the usual range.
• model - optional cdlModel. If omitted, the node returns 0.0 and clips nothing.

The output is norm, a FLOAT - the measured gradient norm before clipping.

Where you'd use it

In any ComfyDL training loop that's showing the classic RNN/Seq2Seq symptoms - loss that oscillates wildly or jumps to NaN. Recurrent networks are notorious for this, and d2l's own RNN chapters use gradient clipping as a matter of routine. If you're training with CdlSgdStep, wire the model in, set theta to 1.0, and log the norm output so you can actually see the spikes you're clipping.

Installing it

It ships with ComfyDL, one install for all 106 nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL
pip install -r ./ComfyDL/requirements.txt

Restart ComfyUI, or search "ComfyDL" in ComfyUI Manager.

Gotchas

• No model, no work. It's an optional input with a silent no-op: leave it disconnected and you get norm = 0.0 back. If your clip "doesn't do anything," check the model is actually plugged in.
• Grads must already exist. Run this before backward and there's nothing to clip - the parameter list comes up empty and it returns 0.0.
• The norm output is the pre-clip norm, so a value above theta doesn't mean the clip failed - it means it just did its job.
• ComfyDL is an educational pack with a tiny footprint, so there's no community lore to fall back on; the README and the d2l source this ports from are your references.