NNT Inference

Training is only half the story - eventually you want to use the model. NNT Inference is the node that takes a compiled MODEL, feeds it new data, and returns predictions, confidence scores, and a little metrics summary. It's the payoff node: the one you wire up when you want to see whether that thing you trained on 1000 MNIST digits can recognize one you just drew.

What it actually does

It runs the model in eval mode with gradients off (no accidental training during inference), then shapes the output to your needs. The output_type dropdown is the core choice: raw gives you the unprocessed logits, probabilities applies softmax so each class gets a 0–1 score that sums to 1, and class_predictions argmaxes straight to "this is class 3." return_confidence gives you the model's confidence (the max probability per prediction) as a separate tensor. Outputs: output_tensor (predictions), confidence_scores (if enabled), inference_info (a STRING with sample count, processing time, mean confidence, output shape), and metrics (a DICT with timing and confidence stats).

Inputs that matter

• MODEL - the compiled or loaded model.
• input_tensor - the data to run, e.g. from NntImageToTensor or a data loader.
• mode - single (just the sample at index), batch (a window starting at index, up to batch_size long), or all (the whole tensor). Iterating a batch at a time is how you handle more data than fits in memory at once.
• output_type - raw / probabilities / class_predictions, as above. probabilities is the sensible default.
• device - cpu or cuda (default). The node gracefully falls back to CPU if CUDA isn't available, so you don't have to think about it on a no-GPU box.

Optional extras worth knowing: index_list accepts something like [0,1,4,7] to run a specific handful of samples in batch mode, and preprocessing (None/normalize/standardize) applies min-max or z-scaling to the input before the forward pass - handy when your training data was normalized and the live input isn't.

Gotchas

Shape matters more here than anywhere. mode: single slices input_tensor[index:index+1] and expects your model to accept it; if your compiled model auto-flattens or the channel layout is wrong, you'll get a shape error that's really a "go back and check NntInputLayer" error. And note the confidence output: with return_confidence on, probabilities mode gives meaningful confidence values; in raw mode confidence is just set to ones, so don't read it as real confidence there. If inference errors, inference_info carries the message - read it before hunting through the graph.

Installing NNT

Part of inventorado/ComfyUI_NNT. ComfyUI Manager (search "ComfyUI Neural Network Toolkit") or:

cd ComfyUI/custom_nodes
git clone https://github.com/inventorado/ComfyUI_NNT.git
cd ComfyUI_NNT
pip install -r requirements.txt

Restart ComfyUI after. The requirements are a heavy scientific stack (torch, numpy, scikit-learn, pandas, transformers, shap), so budget the first install. The pack's "load and infer MNIST" workflow in workflows/ shows the exact intended chain, and it also wants ComfyUI-Jjk-Nodes for text display.