Pt Evaluate Classification Model

Training a model is only half the job - you need to know if it actually learned anything. PtEvaluateClassificationModel runs a trained model over a test set and hands you the scoreboard: accuracy, precision, recall, F1, and how many samples it looked at. It's the node that turns "I trained something" into "my model hits 94% on CIFAR-10," which is exactly the kind of number that tells you whether to keep tuning or ship it.

It's part of ComfyUI-Pt-Wrapper, Hide Inada's no-code PyTorch lab for ComfyUI - a spin-off of his ComfyUI-Data-Analysis that grew into ~200 nodes for building, training, and evaluating models entirely in the graph. This node is the payoff of that whole story. The author's own example workflows lean on it to verify the ResNet-on-CIFAR-10 runs (94%+ validation accuracy) and the dog-vs-cat classifier.

How it works

The node runs your model in inference mode (torch.inference_mode()), iterates your test data_loader in batches, takes argmax over the model's outputs to get predicted classes, and compares against the labels. The metrics come from scikit-learn - accuracy plus macro-averaged precision, recall, and F1 - which is why scikit-learn is a hard dependency of the pack. One detail worth knowing: the code deliberately calls model.train(False) instead of model.eval() because ComfyUI's security checker blocks eval() calls. Same effect, different spelling.

Inputs that matter

• model - a PTMODEL you trained with the pack's trainer nodes (or loaded with PtLoadModel). It must output class logits - the node does argmax(dim=1) on the raw output.
• data_loader - a PTDATALOADER over your test/validation set, with the same class structure as training.
• use_gpu - a boolean, default off. Flip it on if you want the evaluation to run on CUDA; leave it off and it runs on CPU, which for a small test set is often faster than the transfer overhead.

The five outputs are accuracy, precision, recall, f1 (all FLOAT) and num_samples (INT). Wire the floats into a text node to read them, or into comparison nodes to gate a workflow on whether the model cleared a threshold.

Installing it

ComfyUI Manager → Install Custom Nodes → search "Pt Wrapper" → install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/HowToSD/ComfyUI-Pt-Wrapper

Then restart ComfyUI. This is the heaviest install in the pack: requirements pull matplotlib, pandas, scipy, scikit-learn, transformers, datasets, accelerate, peft, and a pinned gensim==4.3.2. Give the first launch time to install everything. Some of the author's example evaluation workflows also reference nodes from his other pack, ComfyUI-Data-Analysis - for those specific example JSONs you'll need that too, but not for this node on its own.

Common issues

The big one: class balance. Precision, recall, and F1 are macro-averaged, so with a heavily imbalanced test set, the numbers can look bad even when accuracy looks fine - or vice versa. That's informative, not a bug. Also check the use_gpu flag matches your setup: a model trained on GPU evaluated with use_gpu off will happily run on CPU, just slower. And if you get a shape error, it's almost always your model outputting something other than raw per-class logits - the node assumes argmax(dim=1) over class scores, so skip any softmax you might have added in training.