YuE2 Training Dataset (audio folder)

The dataset step, minus the dataset tooling

Every image LoRA workflow starts with a folder of pictures and then a whole second conversation about captioning, bucketing and resolution. The music version is refreshingly cruder: you have songs, they're already the right length and the right sample rate, and the only real work is turning them into latents the trainer can chew on.

YuE2TrainingDataset is that work. Point audio_folder at a directory of tracks, pick the YuE2 checkpoint, hit Queue, and you get a YUE2_TRAIN_DATASET object to wire into YuE2LoRATrainer. There's no manifest, no .jsonl, and no caption-everything-first ritual - captions are optional .txt files sitting next to the audio.

What it actually does when you queue it

The source, not the marketing: it scans the folder for .mp3, .wav, .flac, .ogg and .m4a, loads each file with torchaudio and falls back to soundfile if torchaudio can't decode it (that's what that dependency is for - some mp3/flac builds), resamples to 48 kHz, and encodes with the checkpoint's built-in VAE into [frames, 64] latents at 25 fps. So 10 seconds of audio is 250 latent frames.

The encode is done in overlapping 30-second chunks with a 2-second overlap that gets trimmed, which is why a 6-minute track doesn't blow up your VRAM. Latents land as float16 .npy files in the cache folder - temp/yue2_latents unless you set cache_folder.

Then clips: the node cuts each file into consecutive clip_seconds chunks and drops the leftover tail. A 25-second track with clip_seconds = 10 becomes two clips and 5 seconds you paid to encode and will never see again. If every file is shorter than one clip, you get a hard error telling you to lower clip_seconds.

The inputs that matter

• checkpoint - the native all-in-one YuE2 file in models/checkpoints; its VAE does the encoding. Use the same bf16 file the trainer and your generation workflow use. One download covers all three.
• audio_folder - an absolute path as a plain string. This is where almost everyone's first error comes from: relative paths and Windows paths typed wrong both fail with "Training folder not found."
• clip_seconds - default 10, range 1–60. 10 is the sweet spot; 6–8 is your VRAM escape hatch if training later OOMs, and remember every clip is a training sample, so smaller clips also means more samples from the same music.
• caption_mode - txt_file reads a same-named .txt next to each song (empty caption if missing), default applies default_caption to everything, none trains on the trigger word alone. For a style LoRA on a consistent set, none or a single default caption is genuinely enough.
• default_caption - only used in default mode. Describe style, instruments, voice.
• cache_folder and force_reencode - the cache is keyed on file path, size, mtime and clip_seconds, so swapping a song under the same name usually re-encodes on its own. force_reencode is the override for when the mtime didn't change (a copy that preserved timestamps, a network share) or when you just want certainty. Reruns with nothing changed are near-instant, because nothing is re-encoded at all.

Outputs

dataset goes into the trainer's dataset input - that's the whole point of the node. summary is a plain STRING reporting source file count, clip count and clip length, total minutes, how many clips got a .txt caption, the cache directory, and a per-file clip breakdown. It's the cheapest sanity check in the pack: read it before a 3000-step run and notice the song that contributed zero clips.

Installing, once for the whole pack

ComfyUI Manager → search ComfyUI-YuE2-Trainer, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Starnodes2024/ComfyUI-YuE2-Trainer.git
cd ComfyUI-YuE2-Trainer && python_embeded/python.exe -m pip install -r requirements.txt

That pulls in soundfile (fallback decoder, usually already there) and matplotlib (curve node only). Download yue2_3b_bf16.safetensors (~7.8 GB) from Comfy-Org/YuE2 into ComfyUI/models/checkpoints/, restart ComfyUI, and the nodes appear under YuE2/Training. Not the INT8 file - quantized weights can't be trained.

Where it goes wrong

• Wrong or relative audio_folder → Training folder not found. Absolute path; forward slashes are fine on Windows.
• "No audio files found" in a folder you know has audio → the scan reads only that directory, not subfolders, and only those five extensions. Nested album folders are the usual culprit. And because clips are cut sequentially, a track shorter than clip_seconds is skipped with a console warning rather than an error - check the summary output.
• Encoding is the slow part, not the clever part. A 20-minute dataset is real wall-clock time on the first run and near-zero afterwards. Don't delete the cache folder between runs out of tidiness.

Two housekeeping notes for the whole pack: 24 GB of VRAM is recommended for what comes after this node, and YuE2's weights are CC BY-NC 4.0, so the LoRA you train and the music you make with it are non-commercial too. Only point this at audio you have the rights to use - the node will happily encode anything you hand it.