ComfyUI Node

Sp Load Model

Load a SentencePiece tokenizer model — the text-tokenization on-ramp

By HowToSD·Created about a year ago·Updated about a year ago· 7
Sp Load Model
    • spmodel
    model_pathspiece.model

    Sp Load Model loads a SentencePiece tokenizer model into the graph, giving you the SPMODEL object that every text-tokenization step in this pack feeds on. If your workflow is doing text classification - IMDB sentiment, say - this is the node that turns raw sentences into the token IDs a model can train on. Without it, the whole text side of ComfyUI-Pt-Wrapper has nothing to chew on.

    It's part of ComfyUI-Pt-Wrapper, the ~200-node pack that brings PyTorch training into ComfyUI. The Sp prefix is the SentencePiece side: a small family that loads the tokenizer model (SpLoadModel) and then uses it to encode text (PtvHfDatasetWithTokenEncode and friends) before data hits a dataloader and a training node.

    How it works

    SentencePiece is Google's subword tokenizer - the kind of model behind T5 and friends. It splits text into a fixed vocabulary of word pieces rather than whole words, so it handles out-of-vocabulary words gracefully. Under the hood this node is:

    sentencepiece.SentencePieceProcessor(model_file=...)
    

    which wraps a .model file into a ready-to-use processor. The path resolution matters: model_path is relative to the pack's models/sentencepiece/ directory, not your ComfyUI root. The default spiece.model points at a model file the pack actually ships - it's bundled inside the repo, sourced from google-t5/t5-base (Apache 2.0 licensed) with a vocabulary of 32000 tokens. That's a genuinely nice touch: you can run the text workflows out of the box with zero model downloads. If you want a different tokenizer, drop a .model file into that directory and type its name here. Because the node's IS_CHANGED flag always forces a reload, picking a new model path takes effect without a restart.

    Inputs and outputs

    • model_path (STRING, default "spiece.model") - filename inside models/sentencepiece/. The bundled file already matches the default.
    • spmodel (SPMODEL) output - the loaded processor, wired into tokenization/dataset nodes.

    Installing the pack

    cd ComfyUI/custom_nodes
    git clone https://github.com/HowToSD/ComfyUI-Pt-Wrapper
    

    Restart ComfyUI. Or ComfyUI Manager → search "ComfyUI-Pt-Wrapper" → install. It's registered on the Comfy Registry, and the pack's requirements include sentencepiece itself, so Manager installs it along with pandas, scikit-learn, transformers and the rest. The key difference from the pure-tensor nodes: here there is a model file, and it's included - no separate download step, which is the part that usually trips people up on other text packs.

    Common issues

    • "Model file not found." The path is relative to the pack's models/sentencepiece/ folder. If you point model_path at an absolute path or a file in the wrong directory, it won't resolve. Keep the file in that folder and use just its name.
    • Vocabulary size mismatch. If you swap in a different SentencePiece model, its vocab size must match what your embedding layer expects. The bundled T5 model is 32000 - the pack's own transformer-from-scratch guide tells you to set the embedding layer to exactly that.
    • Missing sentencepiece dependency. If you installed the pack by hand and skipped requirements, this node fails at import time. Install the pack's requirements.txt (or use Manager, which handles it).

    It's the quiet on-ramp for all of the pack's text work - one bundled model, one node, and suddenly your sentences can become training data.

    CategoryData Analysis

    Inputs (1)

    NameTypeDefaultDescription
    model_pathSTRINGspiece.model

    Outputs (1)

    NameTypeDescription
    spmodelSPMODEL