NNT Define Activation Layer

Activation functions are where a network learns to be nonlinear, and this node is where you stick one in your layer stack. NntDefineActivationLayer appends a standalone activation to your LAYER_STACK - useful when you want an activation between layers without baking it into a Dense or Conv node.

The dropdown has 28 entries (ReLU, GELU, LeakyReLU, SiLU, Tanh, Sigmoid, Softmax, the whole torch.nn shelf). The honest version of this article is: you'll use ReLU (hidden layers), maybe LeakyReLU or GELU if you're curious, Softmax or Sigmoid at the output for classification, and never touch the rest. The menu is exhaustive because the pack's goal is teaching, and part of teaching is letting you click the weird ones.

The inputs

• activation_type (default ReLU) - the function itself. None inserts an identity.
• inplace (default False) - modify the input tensor in place to save memory. Fine for ReLU, risky if you're inspecting tensors downstream; leave it off unless you know why you want it.
• negative_slope (default 0.01) - the LeakyReLU slope, only relevant if that's what you picked.
• num_parameters (default 1) - for PReLU, the number of learned slope parameters.
• alpha (default 1) - for ELU/CELU-style activations.

Output: LAYER_STACK, chained forward as usual.

How it works

Like every define node, it records a dict - {'type': 'Activation', 'activation_type': 'ReLU', ...} - onto the stack. Here's the honest part, and it's the same caveat that runs through this pack's fancier layers: as of the current source, the compile loop in NntCompileModel materializes Conv, Linear, Flatten, Reshape, pooling, and norm layers, but a standalone Activation layer isn't in that dispatch list yet. The activation support that does run lives inside the Dense and Conv define nodes, which apply activation_function themselves. So if you build a stack with a standalone Activation node and compile it, it may not show up in the model.

That's a "work in progress" thing, not a broken thing - the author's README says the toolkit is a learning tool under active development. If you're teaching, this node is still great for understanding the stack structure; if you need the activation to actually execute, set it on the Dense/Conv node instead.

Common issues

• Activation not in your model - see above; use the activation field on Dense/Conv nodes for things that must run today.
• In-place conflicts - an inplace=True activation can trip up autograd on tensors shared elsewhere in the graph. If you get "a leaf Variable that requires grad is being used in an in-place operation," flip it off.

Install

Pack-level, nothing special:

cd ComfyUI/custom_nodes
git clone https://github.com/inventorado/ComfyUI_NNT.git
cd ComfyUI_NNT
pip install -r requirements.txt

or ComfyUI Manager → "ComfyUI Neural Network Toolkit NNT", then restart. That's it - the activation shelf comes with the pack.