MiniMax H3 Attention Parallel

MiniMax H3 is a 33B omni-modal video model. That size is the problem: you need a big card for the weights, and ComfyUI's usual multi-GPU story doesn't get much out of your second and third ones - weight-splitting packs like ComfyUI-Mesh shuffle layers around and pay for it in transfers. This node does something narrower, and for this one model, genuinely clever.

What the node is

It's a MODEL patcher, not a sampler and not an offloader. It clones the model you hand it, swaps the attention implementation for a head-parallel one, and returns the clone. Your H3 weights, text encoder and VAEs never move. The helper GPUs get packed INT8 Q, K and V head slices, do the attention math there, and hand back BF16 results.

Which is why it needs two GPUs minimum to do anything. The model GPU stays the model GPU; the helpers are activation-only, measured at 0.78 GiB each on a four-way split, with no model weights on them at all. Use the cards you have as attention compute, not as parking for 33B of weights.

How the head split actually works

H3 has 56 attention heads, and this node is exact. Each attention call quantizes Q/K/V to INT8 through Comfy Kitchen's prequantize_int8_attention, then splits them into balanced head ranges (28/28, 19/19/18, 14/14/14/14). Every helper runs int8_attention_from_prequantized on its own shard, on its own CUDA stream, and the results copy back into the right head slots of the root tensor. All four device counts produced byte-identical decoded video and audio in the author's test, so devices changes how long a render takes, not what comes out.

That's why it can be exact: attention is head-independent, so slicing heads needs no cross-device reduction. You pay in peer copies instead, so bidirectional CUDA peer access is a hard requirement - the code checks torch.cuda.can_device_access_peer both ways and refuses any device that fails.

The inputs you actually touch

• model - the MODEL wire from your diffusion-model loader. If it isn't a MiniMax H3 model, the node stops immediately with a requires a MiniMax H3 model error rather than silently doing nothing. Wire it before the guider/sampler so the sampler sees the patched model.
• devices - disabled (the default), auto, 2, 3, or 4. This counts the model GPU. 4 means one model card plus three helpers. disabled is a no-op passthrough; auto takes up to four suitable visible GPUs and never fails the run.
• min_sequence_length - default 32768, auto only. Below that packed token count, auto uses one GPU. It's advanced-marked for a reason: a 640x1152, 158-frame run packed to [1, 56, 93312, 128], roughly three times the default threshold, so short clips sensibly stay single-device.

The single output is model. Nothing else, no second socket to wire.

Install

There's no pip step - the pyproject declares zero dependencies, and the README says it uses the Comfy Kitchen version that shipped with your ComfyUI.

cd ComfyUI/custom_nodes
git clone https://github.com/AesSedai/ComfyUI-MiniMaxH3-Parallel.git

Restart ComfyUI after. Manager can install the same URL through Install via Git URL; the pack isn't in the public registry listing yet, so searching the Manager store may come up empty. Requirements aren't small: ComfyUI 0.33.0 or newer, Comfy Kitchen 0.2.31 with CUDA INT8 attention, two to four visible NVIDIA GPUs with peer access, and an H3 model on ComfyUI's native integration. CUDA only, no ROCm path.

Then launch with every participating GPU visible and the INT8 attention path switched on:

CUDA_VISIBLE_DEVICES=0,1,2,3 python main.py --use-ck-attention

What it buys you

The four-GPU reference run cut denoising from 153.78 s to 76.01 s - 2.02x on the denoiser, 1.39x end-to-end (224 s → 161 s) at four steps, 640x1152, 158 frames, four RTX PRO 6000s. That gap is the honest headline: conditioning and VAE decode stay single-device, so they become the bottleneck as you add cards. Attention alone went 534 ms to 197 ms at the same packed shape, so 2.71x is the ceiling and your end-to-end gain lands well under it.

Where people get burned

The failure I'd expect first is Comfy Kitchen INT8 attention is unavailable on cuda:0 - the node's own check, and it usually means you forgot --use-ck-attention, or your Comfy Kitchen build disabled its CUDA/Triton backend for your card. The community has hit exactly that: backends silently switching off on newer CUDA and Blackwell setups is a recurring thread. Check from your ComfyUI venv:

python -c "import comfy_kitchen as ck; ck.list_backends()"

Then peer access: explicit counts fail early naming which devices were rejected and why, while auto just quietly uses fewer, which reads as "no speedup" instead of an error. Also, attention masks and grouped-query head mismatch are unsupported by design, and the adapter is tested against Comfy Kitchen 0.2.31 specifically - retest against a stock run if you bump it. And from the README rather than from bugs: don't bolt Torch Compile Model onto this graph. About 1.5% in steady state, and it drifted off the eager trajectory.

The bigger caveat: with one GPU, this node is a passthrough with extra steps. Leave devices on disabled and spend your time on the quant ladder instead.