VibeVoice LoRA

If you've used LoRAs for images, you already know the shape of this one: a small adapter that steers a big model toward something specific without retraining the whole thing. "VibeVoice LoRA" is exactly that, but for a TTS model - a config node that loads a voice LoRA and hands it to the speaker nodes so you get a consistent, reusable voice instead of whatever the reference sample happens to produce today.

This is the node you reach for when cloning gets you most of the way but not all the way. Voice cloning conditions on a reference sample each run; a LoRA bakes a voice in. Same narrator across a whole series, a character voice you keep coming back to, a voice tuned away from the model's default cadence - those are LoRA jobs. It's also the answer to "why does my character sound different every session."

How it works

A VibeVoice LoRA is a low-rank adapter that mostly steers the LLM half of the pipeline - the part that decides what gets spoken and how. But the pack supports adapters for every component: the language model, the diffusion head, and the acoustic and semantic connectors. That's why this node has four use_* toggles instead of one - you can apply just the LLM adapter, just a diffusion head swap, or any combination, and most training runs will ship only some of them.

• lora_name - dropdown of folders in ComfyUI/models/vibevoice/loras/. "None" disables and runs the base model.
• llm_strength (0.0–2.0, default 1.0) - how hard the LLM adapter pushes. 1.0 is the trained strength; drop it for subtlety, push it for a stronger voice.
• use_llm / use_diffusion_head / use_acoustic_connector / use_semantic_connector - which components to apply, if the adapter has them.

The single output is lora (a LORA_CONFIG), wired into the lora input on either the Single or Multiple Speakers node. That's the whole graph: [VibeVoice LoRA] → [Speaker node].

Where the LoRAs live

Drop adapter folders in ComfyUI/models/vibevoice/loras/:

ComfyUI/models/vibevoice/loras/my_voice/
├── adapter_config.json
├── adapter_model.safetensors
├── diffusion_head/          # optional
├── acoustic_connector/      # optional
└── semantic_connector/      # optional

Required files are adapter_config.json plus the weights. Each adapter adds only ~100–500 MB of memory overhead, which is the whole point.

To train your own, use the official VibeVoice fine-tuning repo (voicepowered-ai/VibeVoice-finetuning) - parameter-efficient fine-tuning with adjustable rank and scaling, which is the standard LoRA training flow covered in our LoRA training notes. One warning that carries over from image LoRAs: adapters are architecture-locked. A VibeVoice LoRA only runs on VibeVoice, and the pack checks for model/LoRA mismatches and refuses to load incompatible combos with a clear error.

Install and gotchas

Same pack, same install as the speaker nodes - Manager search "VibeVoice" or git clone https://github.com/Enemyx-net/VibeVoice-ComfyUI into custom_nodes/, restart, done. There's no separate setup for the node itself; the folder structure is the setup.

Where people get burned:

• Only "None" in the dropdown: the loras/ folder is empty or in the wrong place. It must be under ComfyUI/models/vibevoice/, not models/loras/.
• LoRA loads, then fails to apply: transformers newer than 4.51.3 isn't guaranteed to work with the LoRA loader. The fix the README suggests is explicit: pip install transformers==4.51.3.
• Strengths above 1.0 can push a voice into uncanny territory fast. Start at 1.0, nudge by 0.1.
• Memory: if you're already near your VRAM limit, remember LoRAs ride on top of an 18 GB model - use a quantized base if the Q8/Q4 versions are available.

One more thing worth knowing: the LoRA node doesn't generate anything itself. It's a pure configuration node, so a workflow with just this node and nothing wired into a speaker node will "run" and do nothing visible. That's expected - it's the setup step, not the payoff.