Nodes/ComfyUI-TITrain/Textual Inversion Training SD1.5
ComfyUI Node

Textual Inversion Training SD1.5

Train an SD1.5 embedding from inside ComfyUI

By mbrostami·Created 2 years ago·Updated about a year ago· 13
Textual Inversion Training SD1.5
  • images
    train_dirtrain
    pretrained_model_namerunwayml/stable-diffusion-v1-5
    learnable_propertyobject
    placeholder_tokencat-toy
    embedding_namecat-toy
    initializer_tokentoy
    resolution512
    max_train_steps500
    learning_rate0.0005
    mixed_precisionno
    validation_steps0
    num_vectors1
    batch_size1
    gradient_accumulation_steps4
    seed0
    prompt_extra

    Textual inversion is a 2022 thing, and this node is unapologetically 2022: it trains an SD 1.5 embedding, gives you a progress bar, and drops the finished .safetensors straight into ComfyUI's models/embeddings folder. No separate trainer, no command line, no leaving the graph.

    Why would you bother in 2026, when everyone trains LoRAs? Because embeddings are still genuinely good at one job: a 10–80 KB file that teaches the CLIP text encoder a single concept - a pose, a style, a negative - without touching a single model weight. LoRA is the better tool for character likeness, full stop. But if you're working on an SD 1.5 or SDXL-lineage base and you want something tiny, training-side simple, and locked to your own trigger word, this is the one ComfyUI node that still ships the whole pipeline.

    How it works

    The node is a thin wrapper around a modified copy of HuggingFace's diffusers textual_inversion training example. When you hit run it does three things:

    1. Unloads everything you have loaded - it walks ComfyUI's loaded models and frees the VRAM, because training and generating don't share well.
    2. Writes your input images to disk - the images tensor you feed in gets saved as PNGs under ComfyUI/output/<train_dir>/, which becomes the training dataset. Wire in a batch of Load Image nodes (or anything that outputs an IMAGE) and you're done; no separate folder prep.
    3. Shells out to accelerate launch via pexpect, running the real training script and parsing its -- STEP lines into ComfyUI's progress bar.

    The training itself is classic textual inversion: it adds your placeholder token (rendered in the prompt as <cat-toy>) to the tokenizer, seeds those vectors from the initializer_token - a real word whose meaning is close to your concept - and trains the vectors against the frozen model. num_vectors lets you give the concept 1–15 vectors instead of one, which buys more capacity at the cost of a bigger file. First run downloads runwayml/stable-diffusion-v1-5 from the HuggingFace Hub (several GB), which is also the biggest gotcha: this is a diffusers-format model, not your ComfyUI checkpoints folder.

    The inputs that actually matter

    All fifteen inputs are required, but a beginner sets five:

    • placeholder_token - the new word, e.g. cat-toy. It becomes <cat-toy> in your prompts. Keep it to one or two real-ish words.
    • initializer_token - a seed word close in meaning, like toy for cat-toy. The script's default is a decent starting point.
    • learnable_property - object for a thing, style for an aesthetic. Pick wrong and the whole run is wasted.
    • max_train_steps - the volume knob. 500 default; the diffusers rule of thumb is ~100–200 steps per image, and the node caps out at 8096. Overcook it and the embedding just memorizes your dataset.
    • prompt_extra - one fragment per line, appended to the training prompts. The author's framing is the right one: put in the parts you don't want the embedding to absorb - white background, close up, tattoo - so the model attributes them to the prompt words instead of your vector.

    learning_rate (0.0005 default), gradient_accumulation_steps, batch_size and mixed_precision are the advanced knobs; on an 8 GB card leave them alone. The node has no outputs - it's an output node, it does its work and returns nothing.

    Install

    The README points you at the diffusers textual-inversion example install first, which is the honest version of the requirements: this needs diffusers, accelerate, transformers, xformers and friends in the same Python environment ComfyUI runs from. The pack's own requirements.txt (accelerate, torchvision, transformers, ftfy, tensorboard, Jinja2, pexpect) covers most of it.

    cd ComfyUI/custom_nodes
    git clone https://github.com/mbrostami/ComfyUI-TITrain
    cd ComfyUI-TITrain
    pip install -r requirements.txt
    

    Restart ComfyUI, or grab it via ComfyUI Manager by searching "TITrain". Either way, if training dies with an import error, the fix is installing the diffusers stack into your ComfyUI venv - the most common failure this pack has.

    Where the output goes

    Your embedding lands at ComfyUI/models/embeddings/<embedding_name>.safetensors - note embedding_name is what it's saved as, distinct from placeholder_token. Refresh, drag in a Load Textual Inversion Embedding node, wire it into the CLIP input of your text encoder, and prompt with <cat-toy>. Intermediate checkpoints go to ./save/checkpoints/<placeholder_token>/, and the node resumes from the latest one if you re-run, which makes iterating on step count cheap.

    One warning before you invest an afternoon: the result is married to SD 1.5's CLIP encoder. It won't load on SDXL, and it definitely won't mean anything on Flux or any LLM-encoder model - your UI won't warn you either, it just silently ignores it. This node is for the people still living on SD 1.5 checkpoints.

    Categorytrain

    Inputs (17)

    NameTypeDefaultDescription
    imagesIMAGE
    train_dirSTRINGtrain
    pretrained_model_nameSTRINGrunwayml/stable-diffusion-v1-5
    learnable_propertySTRINGobject
    placeholder_tokenSTRINGcat-toy
    embedding_nameSTRINGcat-toy
    initializer_tokenSTRINGtoy
    resolutionSTRING512
    max_train_stepsINT5001–8096
    learning_rateFLOAT0.0005
    mixed_precisionCOMBOno3 options: no, fp16, bf16
    validation_stepsINT00–2048
    num_vectorsINT11–15
    batch_sizeINT1
    gradient_accumulation_stepsINT4
    seedINT00–18446744073709550000
    prompt_extraSTRING

    Outputs (0)

    No outputs