Realtime LoRA Trainer (Wan 2.2 - Musubi Tuner)

What it is

MusubiWanLoraTrainer trains a character or subject LoRA for Wan 2.2 - Alibaba's video model and, since Alibaba stopped shipping open weights after 2.2, still the base most of the local video ecosystem is built on - using Musubi Tuner, which is Kohya's own tool for exactly this job (its README covers Wan, Hunyuan Video, and a growing list of others despite the "video trainer" reputation being older than its current scope). This node trains on single frames, not video clips, which matches what the community has actually found works: character LoRAs trained on stills generalize fine to motion, and stills are dramatically cheaper to gather and caption than clips.

The reason this one has its own node rather than sharing with RealtimeLoraTrainer's Wan option is the training backend - this uses Musubi's block-swap VRAM controls, which get you further on a mid-range card than the AI-Toolkit path does for the same model.

How it works

Wan 2.2 runs a two-pass architecture: a high-noise model that handles early denoising (motion, composition) and a low-noise model that refines detail. You can train a LoRA against either pass alone, or use Combo mode to train one LoRA that works across both. The pack's README flags a deliberate trick here: even when you're training just the High or Low model, the example workflows still wire the LoRA into both models in the graph - just at zero strength for the one you didn't train - specifically to stop ComfyUI from loading the untrained base model into memory before training starts.

Inputs and outputs that matter

• noise_mode - High Noise, Low Noise, or Combo. This is the decision that matters most; Combo gives you one LoRA that works on both passes, at some cost to how specialized it is for either.
• dit_model, vae_model, t5_model dropdowns - populated from your local model folders once you've downloaded the fp16 Wan weights the README points to.
• inputcount plus image_N / caption_N pairs, or images_path for a folder.
• training_steps (500), learning_rate (0.0003), lora_rank (16).
• vram_mode - eight presets from Max (1256px) down to Min (512px), each with an fp8 variant.
• blocks_to_swap - 0 to 40, Musubi's block-offloading dial for trading VRAM against speed.

Output is a single lora_path string, ready for ApplyTrainedLora or the Wan Selective Loader.

Installing it

Same pack, same install:

cd ComfyUI/custom_nodes
git clone https://github.com/ShootTheSound/comfyUI-Realtime-Lora

Restart ComfyUI, then install Musubi Tuner separately and point musubi_path at it. Download the fp16 Wan weights - wan2.2_t2v_14B_fp16.safetensors (High or Low variant), wan_2.2_vae.safetensors, and models_t5_umt5-xxl-enc-bf16.pth - from Comfy-Org or the links bundled in the pack's example workflows.

Common issues

Confusing High/Low/Combo. If your LoRA looks fine on one pass and wrong on the other, you likely trained a single-pass LoRA and applied it everywhere - either train Combo, or train both High and Low separately and load them accordingly.

VRAM pressure. Wan 2.2's 14B model is not small. Raise blocks_to_swap and drop into an fp8 vram_mode preset before assuming your card can't handle it; Musubi's own guidance suggests roughly 24GB VRAM and 64GB system RAM as a comfortable floor for video training.

Wrong Python version. 3.10–3.12 as usual, avoid 3.13.

Underwhelming likeness from a still-image dataset. This is expected and not a bug - training on stills gets you identity, not motion quality. If the character looks right but moves stiffly, that's a base-model and sampler question, not something this trainer fixes.