H3 Modality LoRA Loader

So you've got a MiniMax H3 LoRA trained to make a character look like your friend, you drop it in with ComfyUI's normal LoRA loader, and suddenly the audio in your clip is also "your friend". Or your voice LoRA is quietly restyling the visuals. That bleed is exactly the problem this node exists to fix.

MiniMax H3 isn't three models in a trench coat. It's a 33B omni-modal model that packs text, image, video, and audio into a single sequence and runs them through shared Linear layers. Train a LoRA on those shared layers and it targets everything those layers touch - which is why an H3 LoRA that's supposed to be visual-only still messes with the soundtrack. The H3 Modality LoRA Loader fixes that by loading up to 10 H3 LoRAs in one node and gating each one per modality.

How it works

The node checks that your model is actually a MiniMaxH3Model first - anything else and it raises a ValueError on the spot, which saves you from a confusing silent no-op. Then it wraps every supported Linear layer (qkv_proj, out_proj, fc1, fc2, and the modality-dedicated projections like video_patch_proj and audio_out) with a small additive delta, scaled per row of the packed sequence by a mask. A wrapper on diffusion_model.forward re-derives the packed layout, tags each row as video/text/audio/conditioning, and shares the mask with all the layer wrappers.

The upshot: an audio-only LoRA's delta only lands on rows the model is treating as audio, and a video LoRA stays off your sound. It's not perfect - the author is upfront that self-attention still leaks a little between modalities, just far less than usual. There's also a real performance story: batching beats chaining single LoRA loaders, at the cost of two extra matrix multiplications per affected module (up to 528 on the stock model's 264 modules).

The inputs that matter

• model - the H3 checkpoint, straight from the loader.
• audio / video / text - global strengths for each modality, 0–5, default 1. Set one to 0 to fully mute that modality's LoRA contribution. These are the knobs you'll actually touch.
• stack_data - you never touch this directly. It's a JSON string managed by the node's custom UI: each row is a LoRA plus its strength, serialized so your workflow stays portable.

The single output is a patched MODEL, which wires straight into your sampler, KSampler, or whatever consumes the model next.

Installing it

ComfyUI Manager: search "ComfyUI-H3-Modality-Lora_Loader" (note the underscore) and install. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/Dantemss/ComfyUI-H3-Modality-Lora-Loader

Restart ComfyUI. There are no pip dependencies and no requirements.txt - it's pure Python on top of ComfyUI's own MiniMax support, so you just need a reasonably recent ComfyUI. The H3 weights themselves are a separate, chunky download (~42 GB) and come with a catch the README won't mention: the MiniMax H3 Community License geofences out the US, EU, UK, and South Korea. If you're in one of those regions, running the local weights is technically unlicensed. Worth knowing before you build a whole pipeline on it.

Where people get burned

The README is admirably honest about the rough edges. The drag-and-drop slot UI has several bugs - wrong row dragged, rows dropping above the "add LoRA" button, rows greyed out after dropping. The LoRA refresh button doesn't work and can leave phantom "None" entries in the list; restart the frontend to clear them. And the modality masks are small but can cause VRAM fragmentation, which shows up as OOM or a mysterious slowdown - more likely if you're constantly tweaking strengths. A ComfyUI restart is the reliable fix; setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True before launch may help a little.

One more thing to know: LoRA targets the node doesn't recognize get skipped with a warning in the console, not an error. If your LoRA seems to do nothing, check the terminal - your file may be targeting layers this pack doesn't know about yet. It's a young, single-purpose tool, but for keeping H3 LoRAs out of each other's lanes, there's nothing else that does the job.