ComfyUI Node Runs on cloud
Train LoRA
A ComfyUI node in model/training with 21 inputs and 3 outputs.
Train LoRA
- model
- latents
- positive
- lora
- loss_map
- steps
◄batch_size1►
◄grad_accumulation_steps1►
◄steps16►
◄learning_rate0►
◄rank8►
◄optimizerAdamW►
◄loss_functionMSE►
◄seed0►
◄training_dtypebf16►
◄lora_dtypebf16►
◄quantized_backwardfalse►
◄algorithmLoRA►
◄gradient_checkpointingtrue►
◄checkpoint_depth1►
◄offloadingfalse►
◄existing_lora[None]►
◄bucket_modefalse►
◄bypass_modefalse►
Categorymodel/training
Inputs (21)
| Name | Type | Default | Description |
|---|---|---|---|
| model | MODEL | The model to train the LoRA on. | |
| latents | LATENT | The Latents to use for training, serve as dataset/input of the model. | |
| positive | CONDITIONING | The positive conditioning to use for training. | |
| batch_size | INT | 11–10000 | The batch size to use for training. |
| grad_accumulation_steps | INT | 11–1024 | The number of gradient accumulation steps to use for training. |
| steps | INT | 161–100000 | The number of steps to train the LoRA for. |
| learning_rate | FLOAT | 01e-7–1 | The learning rate to use for training. |
| rank | INT | 81–128 | The rank of the LoRA layers. |
| optimizer | COMBO | AdamW | The optimizer to use for training. |
| loss_function | COMBO | MSE | The loss function to use for training. |
| seed | INT | 00–18446744073709550000 | The seed to use for training (used in generator for LoRA weight initialization and noise sampling) |
| training_dtype | COMBO | bf16 | The dtype to use for training. 'none' preserves the model's native compute dtype instead of overriding it. For fp16 models, GradScaler is automatically enabled. |
| lora_dtype | COMBO | bf16 | The dtype to use for lora. |
| quantized_backward | BOOLEAN | false | When using training_dtype 'none' and training on quantized model, doing backward with quantized matmul when enabled. |
| algorithm | COMBO | LoRA | The algorithm to use for training. |
| gradient_checkpointing | BOOLEAN | true | Use gradient checkpointing for training. |
| checkpoint_depth | INT | 11–5 | Depth level for gradient checkpointing. |
| offloading | BOOLEAN | false | Offload model weights to CPU during training to save GPU memory. |
| existing_lora | COMBO | [None] | The existing LoRA to append to. Set to None for new LoRA. |
| bucket_mode | BOOLEAN | false | Enable resolution bucket mode. When enabled, expects pre-bucketed latents from ResolutionBucket node. |
| bypass_mode | BOOLEAN | false | Enable bypass mode for training. When enabled, adapters are applied via forward hooks instead of weight modification. Useful for quantized models where weights cannot be directly modified. |
Outputs (3)
| Name | Type | Description |
|---|---|---|
| lora | LORA_MODEL | LoRA weights |
| loss_map | LOSS_MAP | Loss history |
| steps | INT | Total training steps |