FunPack Train LoRA

Train a LoRA from a saved training set, on ComfyUI's own trainer. Saves it to the loras folder.

By digital-garbage·Created about a year ago·Updated about 21 hours ago· 20

FunPack Train LoRA

clip
vae
model
audio_vae

lora

◄set_name►

◄namefunpack_lora►

◄usenormal►

◄steps500►

◄learning_rate0►

◄rank16►

◄algorithmLoRA►

◄optimizerAdamW►

◄lossMSE►

◄batch_size1►

◄grad_accumulation1►

◄resolution512►

◄aspect_bucketsfalse►

◄frames33►

◄fps0.0►

◄startstart►

◄trigger►

◄seed0►

◄training_dtypebf16►

◄lora_dtypebf16►

◄gradient_checkpointingtrue►

◄offloadingfalse►

◄bypassfalse►

◄model_name►

◄weight_dtypedefault►

◄compute_dtypedefault►

◄postfix►

◄swap_blocks0►

◄preview_every0►

◄save_every0►

◄resume►

CategoryFunPack/Training

Inputs (35)

Name	Type	Default	Description
clip	CLIP		The text encoder that turns the captions into conditioning.
vae	VAE		The VAE that turns pictures and clips into latents.
set_name	STRING		The training set to learn from.
name	STRING	funpack_lora	File name of the finished LoRA (no extension). An earlier file is never overwritten.
use	COMBO	normal	normal: liked results and your own media. avoid: only the disliked results, for a LoRA you apply at negative strength. all: everything.
steps	INT	5001–100000	How many training steps.
learning_rate	FLOAT	01e-7–1	How big each learning step is. Lower is safer and slower.
rank	INT	161–128	How much the LoRA can learn. Higher holds more detail and makes a bigger file.
algorithm	COMBO	LoRA	The adapter kind. LoRA is the common one.
optimizer	COMBO	AdamW	How the weights are updated.
loss	COMBO	MSE	How the error is measured.
batch_size	INT	11–64	Samples per step. Models that train one sample at a time (H3) lower it to 1.
grad_accumulation	INT	11–256	Steps added up before the weights change: a bigger batch for less memory.
resolution	INT	51264–2048	Training size in pixels per side. Each item is fitted to about this area, keeping its shape.
aspect_buckets	BOOLEAN	false	Keep each picture's own shape (a few size groups) instead of one shape for all. Not used for H3.
frames	INT	331–1025	Frames taken from each clip, for video models. H3 rounds to its own grid (17 x n + 5).
fps	FLOAT	0.00–120	Frames per second to read the clips at. 0 uses each clip's own rate (H3: 24).
start	COMBO	start	Where in a clip to take the frames from.
trigger	STRING		The prefix: text put at the start of every caption, for example a word to call the LoRA up later.
seed	INT	00–18446744073709550000	Seed for the noise and the LoRA's starting weights.
training_dtype	COMBO	bf16	Precision used while training. bf16 is right for nearly everything.
lora_dtype	COMBO	bf16	Precision the LoRA is stored in.
gradient_checkpointing	BOOLEAN	true	Trades speed for a lot less memory.
offloading	BOOLEAN	false	Keeps model weights in system memory while training: slower, for cards that cannot hold the model.
bypass	BOOLEAN	false	Apply the LoRA by hooks instead of changing weights. For quantized models.
model_name	STRING		The diffusion model file to train on, as the loader lists it. The trainer loads it itself, once, with no loader node in front.
weight_dtype	COMBO	default	How the model file's weights are stored when it is loaded here.
compute_dtype	COMBO	default	What the maths runs in when the model is loaded here.
postfix	STRING		Text put at the end of every caption.
swap_blocks	INT	00–128	How many transformer blocks keep their weights in system memory, fetched one layer at a time: slower, and each one frees about a block's weight on the card. 0 keeps the whole model on the card.
preview_every	INT	00–100000	Pause every this many steps so the pipeline can make a preview with the LoRA so far. 0 never pauses.
save_every	INT	00–100000	Save a copy of the LoRA so far to the loras folder every this many steps, named <name>_step<N>. 0 saves only the finished one.
resume	STRING		The name of a paused run to carry on. Empty starts a new run.
modelopt	MODEL		A model already loaded to train on. Leave empty and give model_name to have the trainer load the file itself, once.
audio_vaeopt	VAE		Only for models that learn sound with the picture (H3): turns the clips' sound into latents.

Outputs (1)

Name	Type	Description
lora	STRING	—