VRGDG LTX Audio Video LoRA Train Chunk

This is the heavy one. VRGDG_LTXAudioVideoLoraTrainChunk runs a musubi-tuner training chunk for LTX-2.3 audio-video LoRAs - i.e., it trains a LoRA that captures a voice and a look from short clips that have audio embedded. It's the node VRGameDevGirl uses to build her own LTX character/singer LoRAs, and it's genuinely different from image LoRA training: LTX is a 22B audio+video model, so the trainer has to handle frame buckets, audio buckets, a Gemma text-encoder cache, block swapping for VRAM, and an FP8 base-model path. All of that is surfaced here as knobs, but you don't have to touch most of them.

The workflow is: point it at a folder of short training videos (with audio), pick a training profile (a recipe bundle), and it caches the Gemma text embeddings, trains with musubi-tuner for a chunk of steps, and copies a Comfy-compatible LoRA into your ComfyUI loras folder. It can even apply the new LoRA back onto the MODEL you pass in, so the output model is immediately ready to sample with.

The inputs that matter

You will not tune all 30+ fields. The key ones:

• dataset_videos_dir - folder of training videos (or a parent it organizes into a videos subfolder).
• workspace_dir - working folder for caches, logs, configs, and checkpoints.
• av_profile - the recipe switch, and the most important knob:
  • fast_short_clip (default) - quick test on one short video.
  • step200 - same recipe, 200-step chunk budget, lighter repeats.
  • balanced - stronger middle ground.
  • full_video - trains on the whole clip, slowest.
  • overnight_full_video - tuned for a longer clip at lower res and smaller frame windows so it can run overnight on a midrange GPU.
• use_profile_defaults - on by default; the profile controls the recipe bundle. Turn off only if you want to hand-tune everything.
• resolution_width / resolution_height - 768×512 default; 512 is faster, bigger costs VRAM.
• target_frames - comma-separated frame buckets, default 17,33; the trainer rounds to the required N×4+1 buckets.
• steps_per_run - steps before saving this chunk (100 default).
• total_target_steps - overall budget (400 default).
• network_dim / network_alpha - LoRA rank and alpha, both 16 default. Bump to 32 if the result is too weak.
• blocks_to_swap - transformer blocks offloaded to CPU (2 default). Raise only if you OOM; lower if you have VRAM to spare.
• lora_target_preset - which layers get adapters: t2v (fastest baseline), v2v (adds FFN), audio, or full.
• musubi_root / ltx2_checkpoint / gemma_root - paths to your musubi-tuner install, the LTX-2.3 audio-video checkpoint, and the Gemma model files. The defaults point at A:/MUSUBI/... - that's the author's own machine layout, and you must change these to your paths.
• copy_latest_to_comfy_loras - on by default, copies the trained LoRA into ComfyUI's loras folder.

Outputs: model (base model with the latest LoRA optionally applied via strength_model), plus latest_state_path, log_path, video_filename_prefix, output_name, completed_steps, and total_target_steps for tracking/continuation.

Install & the real requirements

You need three things: the pack, a musubi-tuner-ltx2 install (use VRGDG_MusubiTunerInstaller to get it), and the model files - an LTX-2.3 22B audio-video checkpoint and the Gemma text encoder. Expect tens of gigabytes of downloads and a machine with serious VRAM (the author's overnight_full_video profile exists specifically because midrange GPUs can't do the rest).

cd ComfyUI/custom_nodes
git clone https://github.com/vrgamegirl19/comfyui-vrgamedevgirl
pip install -r requirements.txt

Common issues

• Training is way slower / OOMs immediately: you're not on overnight_full_video or fast_short_clip with a midrange card. Pick the profile that matches your hardware; raise blocks_to_swap only when you actually run out of memory.
• LoRA is too weak: bump network_dim to 32, or total_target_steps up. One 10-second clip can't learn much - num_repeats (8 default) exists to get more signal out of that single clip.
• Captions missing: turn on create_captions with a caption_text fallback, or the trainer has nothing to pair with your frames.
• Text-encoder cache errors: leave clear_memory_before_gemma on - the Gemma cache step is where most failures happen because VRAM is full of other models. gemma_recovery_mode is the experimental fallback if the normal path still fails.

Honest take: this is an expert node for a demanding workflow, not a beginner toy. But it's the closest thing to "train your own LTX audio-video LoRA with one button" that exists, and once you accept that the first run is mostly downloads and the second run is mostly waiting, it works. The musubi-tuner angle also matters: LTX-2.3 LoRA training is where the LTX ecosystem actually lives right now, and this is a well-wired path into it.