Nodes/TTS Audio Suite/๐ŸŽ›๏ธ MOSS Training Config
ComfyUI Node

๐ŸŽ›๏ธ MOSS Training Config

The LoRA hyperparameters for training a MOSS-TTS voice

By diodiogodยทCreated about a year agoยทUpdated 22 days agoยท 1,098
๐ŸŽ›๏ธ MOSS Training Config
    • training_config
    • config_info
    โ—„training_modeLoRA Adapter (Delay 8B)โ–บ
    โ—„epochs3โ–บ
    โ—„max_train_steps30000โ–บ
    โ—„batch_size1โ–บ
    โ—„gradient_accumulation_steps16โ–บ
    โ—„learning_rate0โ–บ
    โ—„weight_decay0.0100โ–บ
    โ—„warmup_steps100โ–บ
    โ—„max_grad_norm0.5โ–บ
    โ—„num_workers0โ–บ
    โ—„mixed_precisionbf16โ–บ
    โ—„gradient_checkpointingtrueโ–บ
    โ—„base_quantizationnoneโ–บ
    โ—„bnb_4bit_compute_dtypeautoโ–บ
    โ—„bnb_4bit_use_double_quanttrueโ–บ
    โ—„save_steps500โ–บ
    โ—„eval_steps500โ–บ
    โ—„log_steps10โ–บ
    โ—„lora_r16โ–บ
    โ—„lora_alpha32โ–บ
    โ—„lora_dropout0.05โ–บ
    โ—„trainable_lora_modulesmlpโ–บ

    This node is the settings panel for a MOSS-TTS training run - epochs, learning rate, LoRA rank, precision, all of it. It doesn't train anything itself; it bundles your choices into a TRAINING_CONFIG that the unified Model Training node reads alongside your prepared dataset. Think of it as the "recipe card" you hand to the trainer.

    Two things to know before you touch a slider. First, this trains a LoRA - a small adapter that sits on top of the frozen 8B MOSS model rather than retraining the whole thing, which is why it's feasible at all on consumer hardware. Second, the author bundled sensible defaults and real tooltip guidance here precisely so you don't have to memorize the raw upstream training flags. For a first run, changing almost nothing is a legitimate strategy.

    How it works

    You set the loop parameters and the LoRA shape; the node packages them. The trainer then runs that many epochs (or steps) over your dataset, saving checkpoints periodically so you can resume or pick the best one. Because it's LoRA, the output is a small adapter file exported into models/TTS/moss_tts/loras/, not a full multi-GB model.

    The inputs and outputs that matter

    You can ignore most of the two dozen fields on a first pass. The ones that actually move the needle:

    • epochs (default 3) and max_train_steps (default 30000) - how long to train. On a small voice dataset a handful of epochs is usually enough; too many overfits and the voice gets brittle. These two together bound the run.
    • learning_rate (default 2e-6) - how big each update step is. The default is deliberately conservative for LoRA-on-a-big-model; crank it up and training can diverge. Change this last, and gently.
    • lora_r (default 16) and lora_alpha (default 32) - the adapter's capacity. Higher rank can capture more but risks overfitting a small dataset and makes the adapter bigger. 16/32 is a reasonable middle.
    • batch_size (default 1) and gradient_accumulation_steps (default 16) - batch 1 with accumulation 16 gives you an effective batch of 16 without needing the VRAM for a real batch of 16. This is how you train an 8B model without a datacenter GPU.
    • base_quantization (none / 4bit_nf4) - set this to 4bit_nf4 to load the base model in 4-bit and slash VRAM use. On a tight card this is often what makes MOSS training possible at all.
    • mixed_precision (default bf16) - leave on bf16 unless your GPU wants fp16.

    Outputs: training_config (TRAINING_CONFIG) to the Model Training node, and config_info (STRING) summarizing your settings.

    How to install it

    Part of TTS Audio Suite. ComfyUI Manager: search TTS Audio Suite, install, restart. Manual:

    cd ComfyUI/custom_nodes
git clone https://github.com/diodiogod/TTS-Audio-Suite.git
cd TTS-Audio-Suite
python install.py

    then restart. On Linux, install portaudio19-dev and libsamplerate0-dev first. This node is pure configuration and downloads nothing - the model comes down when the training run actually starts.

    Common issues & troubleshooting

    Out of memory during training. MOSS is 8B - this is the expected pain point. Set base_quantization to 4bit_nf4, keep batch_size at 1, lean on gradient_accumulation_steps for effective batch size, and make sure gradient_checkpointing (default on) stays on. All four exist to fit a big model on a small card.

    The voice overfit - sounds robotic or memorized. Too many epochs/steps for too little data. Cut epochs, or lower lora_r so the adapter has less capacity to memorize. Small dataset, small adapter, few epochs.

    Training diverges / loss explodes. Almost always learning_rate too high. Return it toward the 2e-6 default; LoRA on a large frozen model wants small steps.

    Only one training mode is available. Correct - as shipped the mode is LoRA Adapter (Delay 8B). The MOSS training pipeline is intentionally scoped to that one target for now; broader options aren't wired in yet.

    CategoryTTS Audio Suite/๐ŸŽ“ Training

    Inputs (22)

    NameTypeDefaultDescription
    training_modeCOMBOLoRA Adapter (Delay 8B)Trains LoRA adapters for the selected MOSS Delay 8B v1.0 or v1.5 base model.
    epochsINT31โ€“1000Epoch count used when max_train_steps is 0.
    max_train_stepsINT300000โ€“1000000Hard training-step cap. Set 0 to use epochs instead.
    batch_sizeINT11โ€“32Per-device batch size. Keep it low on 8B models.
    gradient_accumulation_stepsINT161โ€“256Micro-batch accumulation. This is how you fake a larger effective batch without immediately OOMing.
    learning_rateFLOAT01e-8โ€“1LoRA learning rate. The upstream Norwegian example used 2e-6.
    weight_decayoptFLOAT0.01000โ€“1Standard AdamW weight decay.
    warmup_stepsoptINT1000โ€“100000Warmup step count before cosine decay.
    max_grad_normoptFLOAT0.50โ€“10Gradient clipping. The upstream Norwegian example used 0.5.
    num_workersoptINT00โ€“16Dataloader workers. 0 is the safest default.
    mixed_precisionoptCOMBObf16Accelerate mixed precision mode.
    gradient_checkpointingoptBOOLEANtrueKeep this on for 8B training unless you deliberately want higher VRAM usage.
    base_quantizationoptCOMBOnoneBase-model VRAM strategy. 4bit_nf4 enables QLoRA-style loading of the frozen 8B base model with bitsandbytes. This is the main knob if 24 GB VRAM is still not enough.
    bnb_4bit_compute_dtypeoptCOMBOautoCompute dtype used by 4-bit quantized layers. auto follows the mixed-precision choice. fp32 is safer but costs more VRAM.
    bnb_4bit_use_double_quantoptBOOLEANtrueEnable nested quantization for 4-bit base loading. Usually worth keeping on.
    save_stepsoptINT5000โ€“100000Checkpoint save cadence. 0 disables intermediate checkpoints.
    eval_stepsoptINT5000โ€“100000Validation cadence. 0 disables periodic validation.
    log_stepsoptINT101โ€“10000Console/progress update cadence.
    lora_roptINT161โ€“512LoRA rank.
    lora_alphaoptINT321โ€“1024LoRA alpha.
    lora_dropoutoptFLOAT0.050โ€“1LoRA dropout.
    trainable_lora_modulesoptCOMBOmlpWhich Delay backbone modules receive LoRA adapters. mlp matches the released Norwegian example best.

    Outputs (2)

    NameTypeDescription
    training_configTRAINING_CONFIGโ€”
    config_infoSTRINGโ€”