NNT Text Batch Processor

Neural networks eat numbers, not sentences, and this is the node that does the converting. NNT Text Batch Processor takes a block of plain text, splits it into separate documents, and runs it through a real tokenizer - BERT or DistilBERT - to produce padded, truncated, batched token tensors ready for a training or embedding model. It's the text side of the Neural Network Toolkit's data-loading story, and it's the node you reach for when you want to train something on words instead of pixels.

There's a bigger picture here. The pack's README and the example workflows lean heavily on images (MNIST, CIFAR10), but text is a first-class citizen too - this node is what lets you feed sentences into the same dense/attention stack you built for digits.

How it works

The texts input is a multiline block where documents are separated by a separator line (default ---). The node splits on that separator, then uses Hugging Face's AutoTokenizer to tokenize each batch:

• tokenizer - bert-base-uncased (default) or distilbert-base-uncased. On the first run this downloads the tokenizer from the Hugging Face hub, so you need internet and a moment of patience.
• max_length - every document gets padded (and truncated) to this length, so all rows line up into a rectangular tensor. Default 256, up to 2048.
• batch_size - how many documents get tokenized per pass.
• output_dtype - long by default (token IDs are integers), but you can cast to float variants if a downstream model expects floats.

Outputs: token_batches (the concatenated [num_texts, max_length] tensor of input IDs), num_batches (how many batches were made), and info (a string describing what happened - count, shape, dtype, tokenizer path).

Where it fits and what bites

This is the front end for anything text-classification you build in NNT: tokenize → feed tokens into a model whose input layer matches max_length → train. It also composes with the pack's attention and positional-encoding layers if you want to experiment with transformer stacks on your own text.

The gotchas are real, though. First, the tokenizer download: if the first run hangs or throws an SSL/connection error, it's almost always the Hugging Face download, not your graph - check your network, then re-run. Second, everything gets padded to max_length, so a one-sentence dataset and a 200-word document produce the same huge tensor; keep max_length honest to what your model expects, or you'll be training on padding. Third, this node outputs token IDs - there's no mask output here, so your model just sees padded rows. For a learning toolkit that's acceptable; for real production NLP you'd want attention masks, and this pack won't give them to you.

Install

Pack-level install:

cd ComfyUI/custom_nodes
git clone https://github.com/inventorado/ComfyUI_NNT.git
cd ComfyUI_NNT
pip install -r requirements.txt

Restart ComfyUI, or use Manager → search "ComfyUI Neural Network Toolkit NNT". This node specifically pulls in the transformers dependency (already in the pack's requirements), and the whole requirements list is heavy - torch, sklearn, statsmodels, onnx, shap pinned at 0.41.0, and more. Give the first install time, and expect one HF tokenizer download on first use.