Ptf Leaky ReLU

Ptf Leaky ReLU is the one activation node in ComfyUI-Pt-Wrapper's function family that actually has a knob on it. Like its siblings it doesn't process tensors directly - it emits a PTCALLABLE, the pack's name for a callable function you plug into other nodes. But it's also the only Ptf node here where you set a real parameter: negative_slope.

The context: ComfyUI-Pt-Wrapper is Hide Inada's (HowToSD.com) no-code PyTorch toolkit for the node graph, a spin-off of ComfyUI-Data-Analysis with around 200 nodes spanning tensor math, model construction (ResNet, LSTM, GRU, Transformer), and training. The Ptf nodes are the "activation function" shelf - you grab one and slot it into a model you're assembling.

Why LeakyReLU exists

Plain ReLU does max(x, 0): anything negative becomes exactly zero. That's cheap and it works - until a lot of neurons land in negative territory and stay there. Zero output means zero gradient means those neurons never recover, which is the "dying ReLU" problem. LeakyReLU is the fix: for negative inputs it passes a small fraction through - negative_slope * x instead of 0. Gradients keep flowing, just weakly.

You don't need it for most modern transformer work (GELU/SiLU have taken that job), but for CNNs and MLPs - which this pack's ResNet workflows are full of - it's a legit upgrade over ReLU when you suspect dead units.

What it actually does

The node builds functools.partial(torch.nn.functional.leaky_relu, negative_slope=...) and hands you the callable. The slope you set is baked in. Wire the result into Pt Apply Function (tens + closure → tensor), or into the closure slot of Ptn Model With Closure or Ptn Chained Model to make it the tail of a model.

Inputs and outputs

From info_schema:

• negative_slope - FLOAT, default 0.01, range 1e-6 to 10000, step 1e-5. The multiplier applied to negative values. The standard default is 0.01; you'll rarely want more than 0.1, and above 1.0 it stops being "leaky" and starts being a mirror-flip, so don't go wild.
• Output: PTCALLABLE.

Installing it

It's a Data Analysis category node from ComfyUI-Pt-Wrapper. Install once via ComfyUI Manager (search "ComfyUI-Pt-Wrapper") or:

cd ComfyUI/custom_nodes
git clone https://github.com/HowToSD/ComfyUI-Pt-Wrapper
# then restart ComfyUI

Only torch is needed for this node; the pack's heavier requirements.txt deps (transformers, datasets, peft, accelerate) serve the training and tokenizer side.

Where people get burned

Mostly by the PTCALLABLE concept itself - expecting to feed a tensor straight in and getting nothing to plug into. Pair it with Pt Apply Function when you want output now, or use it as a model closure when you're assembling a network.

One real footgun: the slope applies to negative inputs only. If your tensors come in already shifted so "negative" means something domain-specific, you're not getting the effect you think. And if you're comparing results against a reference architecture that used a specific slope, set it to match - 0.01 and 0.1 are not interchangeable, they'll train differently.