NNT Dataset To Text Tensor

Models don't read words; they read token IDs. NntDatasetToTextTensor is the node that does the translation: it takes a text column from a dataset, runs it through a real HuggingFace AutoTokenizer, and hands you token tensors ready for a model. This is the pack's nod to NLP - the machinery behind the classic BERT-style text classification setup.

The inputs that matter

• dataset - from the pack's loader nodes.
• text_column (default text) - the column of strings to tokenize.
• tokenizer_name (default bert-base-uncased) - any tokenizer name that AutoTokenizer.from_pretrained can resolve. Big gotcha: the first run downloads the tokenizer from the HuggingFace Hub, so you need internet once, and it lands in your HF cache. If your machine is offline or the HF hub is unreachable, this node cannot start.
• max_length (default 512) - the truncation/padding length. BERT's native limit is 512; you rarely need more.
• return_type (default input_ids) - input_ids, attention_mask, token_type_ids, or all. Input IDs are the tokens themselves; attention mask tells the model which positions are real vs. padding; token type IDs distinguish sentence A/B.
• padding (default max_length) - pad to max_length, to the longest in the batch, or not at all. do_not_pad keeps rows ragged, which most models can't handle.
• truncation (True), add_special_tokens (True), pad_to_multiple_of (8) - the standard tokenizer knobs. Multiples-of-8 padding is a Transformer optimization habit.
• requires_grad (default True) - text tensors keep gradients by default here, so if you train an embedding layer on top, they can flow.

Outputs: text_tensor (TENSOR), attention_mask (TENSOR), collated_outputs (DICT from the data collator), and info (STRING).

How it works

It loads the tokenizer, tokenizes the whole column in one call (with padding/truncation/special tokens per your settings), and either returns the raw encodings or runs them through HuggingFace's DataCollatorWithPadding. The pad_to_multiple_of gets handed to the tokenizer so batches come out friendly to fused attention kernels. The detach/clone/requires_grad controls at the end are the pack's standard tensor hygiene - defaults are sensible, leave them alone unless you know why you're touching them.

Common issues

• "No internet" failure on first run - the tokenizer download. Run it once while online, or pre-download the tokenizer with huggingface-cli download bert-base-uncased.
• Sequence too long - if truncation is off and your texts exceed max_length, tokenizers throw. Keep truncation on unless you have a plan.
• Wrong return_type for your model - feed attention_mask to a model that wants input_ids and the shapes won't line up. Match return_type to what the downstream model expects.

Install

Pack-level:

cd ComfyUI/custom_nodes
git clone https://github.com/inventorado/ComfyUI_NNT.git
cd ComfyUI_NNT
pip install -r requirements.txt

or ComfyUI Manager → "ComfyUI Neural Network Toolkit NNT", restart, under NNT Neural Network Toolkit/Data Processing. The transformers dependency (and the tokenizer download) is the heaviest part of this node's footprint.