FunUI Train LoRA
Train on the clips you rated, not a folder you assembled
- clip
- vae
- model
- audio_vae
- lora
Almost every LoRA trainer assumes you're going to spend an afternoon curating a dataset folder: 20–40 images, captioned, bucketed, uploaded to a rented GPU, watched through a loss curve. FunUI's LoRA training add-on takes the opposite angle - you already rated a few hundred of your own generations in the studio, and the ratings are the dataset.
FunUI Train LoRA is the node that does the actual training. It's the least "just try it" node in the pack: it's an add-on you switch on, it only has real mileage on one model, and the README says as much.
What it is
Train a LoRA from a saved training set on ComfyUI's own trainer, and write it into your models/loras folder. The training loop follows musubi-tuner's MiniMax H3 recipe - which layers get a LoRA, per-block checkpointing, noise sampling, loss weighting, optimizer and learning rate - and its block-swap trick, where a number of transformer blocks keep their weights in system memory.
Read that last part as the honest framing: this is not a novel trainer, it's a well-chosen existing recipe with the UI and dataset plumbing wrapped around it. That's usually the right call.
Before the node: the set
set_name is a set you built in the app's LoRA training screen - pick the data, describe each item, done. There's no way to point this node at an arbitrary folder of images; the set is the unit. And use decides what comes out of it:
- normal - liked results plus your own media. The usual one.
- avoid - only the disliked results, for a LoRA you then apply at negative strength. That's the surviving version of the "negative embedding" idea: a LoRA trained on bad output, used negatively. It fails loudly instead of silently, which is more than EasyNegative can say on a modern encoder.
- all - everything in the set.
The inputs that matter
clip and vae are required because something has to turn captions into conditioning and pictures into latents. For an audio-video model, wire audio_vae too or the soundtrack isn't part of what's learned.
For the model itself you have two routes: wire a loaded model, or leave it empty and give model_name - the trainer then loads the file itself, once, with no loader node in front. The second is the one to use when VRAM is tight, because it doesn't leave a second copy of a 33B model hanging around.
The rest is a normal trainer's dials, and the pack's defaults are sane: 500 steps, learning rate 1e-4, rank 16, AdamW, MSE, batch 1, bf16 training and LoRA dtype, gradient checkpointing on. On a video model batch size is 1 by design - H3 trains one sample at a time - and grad_accumulation is how you fake a bigger batch without the memory.
Worth knowing about, because they're what makes a long run survivable:
- save_every - dumps
<name>_step<N>into the loras folder as it goes. Set it. The final epoch is not reliably the best checkpoint, and this is the cheapest insurance in the node. - preview_every - pauses every N steps so the pipeline can make a preview with the LoRA so far.
- resume - the name of a paused run to pick up.
- swap_blocks / offloading - move weights to system memory, slower, for cards that can't hold the model.
- trigger and postfix - text wrapped around every caption, so you have a word to call the LoRA up with later.
Output is a single lora string - the file it wrote. It is not a MODEL, so it doesn't plug into a sampler. You load it with a LoRA loader afterwards, like any other LoRA. And the filename is never overwritten: a second run with the same name doesn't clobber the first.
Installing it
cd ComfyUI/custom_nodes
git clone https://github.com/digital-garbage/FunUI
pip install -r requirements.txt
Then enable it: Settings ▸ Modules ▸ LoRA training. Without that switch the node throws "LoRA training is switched off" the moment you queue. It rides on your existing PyTorch - the pack's other add-ons (spaCy for automatic camera and focus, nvidia-vfx for RTX Super Resolution) are the ones that install extra packages on the next start.
Where people get burned
Enthusiasm vs. reality. The README is explicit that the add-on has been run on MiniMax H3 only. Stable Diffusion 1.5 is marked experimental and LTX 2.x is marked untested across the whole pack. Don't be the first person to discover that training a LoRA on an untested architecture produces eleven hours of nothing.
Expecting an SDXL-style recipe to carry over. It doesn't. Rank, LR and step counts are architecture-specific; the KB's training doc has per-architecture numbers, and the honest summary is "start from the trainer's default". For H3, 16 / 1e-4 / 500 is a starting point, not a law.
A set with nothing in it for the mode you picked. Choosing avoid on a set with no disliked clips gets you a validation error naming the set, which is at least clear. Rate some clips you hated first.
Inputs (35)
| Name | Type | Default | Description |
|---|---|---|---|
| clip | CLIP | The text encoder that turns the captions into conditioning. | |
| vae | VAE | The VAE that turns pictures and clips into latents. | |
| set_name | STRING | The training set to learn from. | |
| name | STRING | funui_lora | File name of the finished LoRA (no extension). An earlier file is never overwritten. |
| use | COMBO | normal | normal: liked results and your own media. avoid: only the disliked results, for a LoRA you apply at negative strength. all: everything. |
| steps | INT | 5001–100000 | How many training steps. |
| learning_rate | FLOAT | 01e-7–1 | How big each learning step is. Lower is safer and slower. |
| rank | INT | 161–128 | How much the LoRA can learn. Higher holds more detail and makes a bigger file. |
| algorithm | COMBO | LoRA | The adapter kind. LoRA is the common one. |
| optimizer | COMBO | AdamW | How the weights are updated. |
| loss | COMBO | MSE | How the error is measured. |
| batch_size | INT | 11–64 | Samples per step. Models that train one sample at a time (H3) lower it to 1. |
| grad_accumulation | INT | 11–256 | Steps added up before the weights change: a bigger batch for less memory. |
| resolution | INT | 51264–2048 | Training size in pixels per side. Each item is fitted to about this area, keeping its shape. |
| aspect_buckets | BOOLEAN | false | Keep each picture's own shape (a few size groups) instead of one shape for all. Not used for H3. |
| frames | INT | 331–1025 | Frames taken from each clip, for video models. H3 rounds to its own grid (17 x n + 5). |
| fps | FLOAT | 0.00–120 | Frames per second to read the clips at. 0 uses each clip's own rate (H3: 24). |
| start | COMBO | start | Where in a clip to take the frames from. |
| trigger | STRING | The prefix: text put at the start of every caption, for example a word to call the LoRA up later. | |
| seed | INT | 00–18446744073709550000 | Seed for the noise and the LoRA's starting weights. |
| training_dtype | COMBO | bf16 | Precision used while training. bf16 is right for nearly everything. |
| lora_dtype | COMBO | bf16 | Precision the LoRA is stored in. |
| gradient_checkpointing | BOOLEAN | true | Trades speed for a lot less memory. |
| offloading | BOOLEAN | false | Keeps model weights in system memory while training: slower, for cards that cannot hold the model. |
| bypass | BOOLEAN | false | Apply the LoRA by hooks instead of changing weights. For quantized models. |
| model_name | STRING | The diffusion model file to train on, as the loader lists it. The trainer loads it itself, once, with no loader node in front. | |
| weight_dtype | COMBO | default | How the model file's weights are stored when it is loaded here. |
| compute_dtype | COMBO | default | What the maths runs in when the model is loaded here. |
| postfix | STRING | Text put at the end of every caption. | |
| swap_blocks | INT | 00–128 | How many transformer blocks keep their weights in system memory, fetched one layer at a time: slower, and each one frees about a block's weight on the card. 0 keeps the whole model on the card. |
| preview_every | INT | 00–100000 | Pause every this many steps so the pipeline can make a preview with the LoRA so far. 0 never pauses. |
| save_every | INT | 00–100000 | Save a copy of the LoRA so far to the loras folder every this many steps, named <name>_step<N>. 0 saves only the finished one. |
| resume | STRING | The name of a paused run to carry on. Empty starts a new run. | |
| modelopt | MODEL | A model already loaded to train on. Leave empty and give model_name to have the trainer load the file itself, once. | |
| audio_vaeopt | VAE | Only for models that learn sound with the picture (H3): turns the clips' sound into latents. |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| lora | STRING | — |