LatentOperationGELU

GELU is the activation hiding inside most of the models you already run - transformer blocks in SDXL, Flux, and their descendants use the GELU family as their default nonlinearity. So there's something quietly appealing about applying GELU to latent space: you're hitting your latent with the exact curve your model's internals are built around. Smooth, non-monotonic for small negatives, and it keeps a sliver of negative information alive instead of hard-zeroing it. If any activation deserves a shot in a latent experiment, it's this one.

It's one of the activation nodes in hnmr293's ComfyUI-latent-ops pack, and it follows the pack's standard pattern: capture a parameter, hand back a deferred operation.

How it works

The node captures alpha and builds a closure that runs torch.nn.functional.gelu(latent * alpha) when applied. The alpha pre-scales the latent before the curve - at the default 1.0 you get plain GELU, and adjusting it changes how fast the curve responds to input. GELU is smooth everywhere: positives pass through nearly linearly, negatives dip slightly below zero before saturating toward zero, and there's no kink at the origin. That slight negative dip is the practical difference from ReLU-style curves - small negatives survive, which can matter when you're preserving texture.

What matters

• alpha - FLOAT, default 1.0, range -10000 to 10000, step 0.0001. Pre-scale of the input.
• op - the only output. A LATENT_OPERATION - a deferred operation, not a result.

The pack-wide caveat applies here too: the op output won't connect to a latent input, and the pack ships no Apply node. You need a consumer that accepts LATENT_OPERATION, or a tiny custom apply node that calls op(latent["samples"]). Without that step, every Operation node in this pack - GELU included - is inert.

Installing it

No deps, no model files - ComfyUI Manager → search "ComfyUI-latent-ops" → install, or:

cd ComfyUI/custom_nodes
git clone https://github.com/hnmr293/ComfyUI-latent-ops

Restart ComfyUI; it's under hnmr/latent_ops.

Troubleshooting

If you were expecting hard ReLU-style zeroing, GELU's preserved negative sliver will surprise you - that's the feature. If you want the cheaper, clamped-version family, the pack's HardTanh and HardSigmoid do bounded squashing instead. And one honest note: GELU is smooth but not bounded - values can still grow - so if you feed the result to a preview that expects a fixed range, a Clamp or NormalizeMinMax after the op is a sensible habit.