H3 MultiStream
Does a Second GPU Actually Help? For MiniMax H3, Yes
- model
- gpus
- MODEL
Ask r/comfyui whether to buy a second GPU and you'll get the answer that's been correct for three years: no, it won't speed up a single render. The test that settled it - two RTX 5080s against one 5090, July 2026 - found that splitting one job across two cards made it slower, transfer and synchronisation overhead eating the gain, and that the only reliable dual-GPU setup is two independent jobs on two cards.
That's true for most models, which is why this pack is interesting. H3 MultiStream gets real acceleration out of a single render: 65.0 s/step to 25.1 s/step across three GPUs, 2.59x, and 4.30x with sparse attention on. H3 turbo, 243 frames at 1344×768, 8 steps, same seed, RTX PRO 4000 Blackwells over PCIe.
Three things make it work where the usual case fails. H3's attention cost grows quadratically with clip length while the exchange between cards grows linearly, so splitting wins on long clips - 1.33x at 192 frames, 1.56x at 362. ComfyUI's core MultiGPU CFG Split doesn't apply, because H3 runs at CFG 1 and there's no second pass to farm out. And the method is sequence parallelism - the Ulysses all-to-all head split Raylight and xDiT implement - not layer-by-layer offloading.
The caveats: on the reference system, PCIe gen3 x8, p2p measured slower than staging through host RAM (189 s against 168 s at 192 frames), and the split is bit-exact against single-GPU ComfyUI only in dense mode.
How the split works
H3 packs text, conditioning, audio and video tokens into one sequence. Each GPU - a rank - owns a contiguous range of tokens and a contiguous group of attention heads, both sized by its share. Equal shares give 28/28 on two GPUs, 18/19/19 on three, 7 each on eight. H3 has 56 heads, so that's the ceiling.
Per block: each rank normalises and modulates its own tokens; everyone all-gathers hidden states so any rank can project all tokens; each rank computes only its own head group by slicing the query/key/value projection rows from the int8 weight, which is what keeps the result exact; attention runs over the full sequence for those heads, then an all-to-all sends each rank every other head group's output for its tokens; finally the output projection and MLP run locally. One thread per rank, ComfyUI's own kernels, hooked in as a DIFFUSION_MODEL wrapper.
The inputs worth setting
model is the patched H3 model, after device selection and any model patches, and the output is a MODEL that goes to Basic Guider and Basic Scheduler. That's the whole interface.
Of the rest, four matter. exchange picks host (pinned-RAM staging, works everywhere) or p2p (direct GPU-to-GPU). On PCIe, keep host - the measured default is the fast one here, and p2p is for NVLink. exchange_chunks pipelines that exchange so one card's upload overlaps its download; 0 is single-shot, 8 measured best (1.12x on the step, 1.41x on the exchange itself). sparse_attention is required if you use ComfyUI's Model Sparse Attention node - a sparse patch replaces the block's whole attention stage and can't be passed through otherwise, so without this switch the node raises.
weight_cache (default on) keeps the block weights in pinned RAM - ~18.5 GB for H3 int8 - and survives prompts and reloads. Leave dynamic_vram at keep and vram_block_cache off: the latter is refused unless dynamic_vram is off for this model, which costs a second load of the checkpoint per process, and neither improved performance on the tested systems.
The optional gpus input takes a H3 MS GPU Set; without it, every visible GPU is used.
Sparse attention isn't a free speedup
This is the part worth reading twice. sol-attn made the step 1.42x faster than dense inside the split. It also produced a different video: mean PSNR 17.7 dB, SSIM 0.666, against the dense render at the same seed across 124 frames. Agreement collapses as motion accumulates, and the visible difference is timing - where a hand is at a given frame. Composition and palette hold.
It's deterministic, so a seed you like reproduces. But you can't recover a dense render by switching sol-attn on afterwards, so choose the mode before you seed-hunt. sparse_vsa is the rollback switch for the vsa method, which needs a checkpoint carrying to_gate_compress - most H3 checkpoints don't have it.
Install
cd ComfyUI/custom_nodes
git clone https://github.com/martonsagi/Comfy-H3-MultiStream ComfyUI-H3-MultiStream
Or ComfyUI Manager, search ComfyUI-H3-MultiStream. Restart; nodes appear under advanced/model, advanced/conditioning and advanced/latent, plus an H3 MultiStream entry in the main menu.
No Python dependencies at all - the pack's pyproject.toml declares none, and torch, psutil, tqdm and aiohttp already ship with ComfyUI. It needs ComfyUI 0.35.0 or newer (native H3, Model Sparse Attention, comfy-kitchen 0.2.33, comfy-aimdo 0.5.3), Python 3.12/3.13, and Linux for the VAE split. The model files are not included. Before you spend a weekend on this, run nvidia-smi topo -m for NV# links (avoid pairs marked SYS) - and read the MiniMax H3 licence: the Community License excludes the EU, UK, US and South Korea from its Applicable Territory, outputs included.
The things that will bite you
unsupported dit patches means a per-block patch replaces the block computation rather than just adjusting transformer_options. Those cannot be split. If it's Model Sparse Attention, switch sparse_attention on.
If you submit through the HTTP API, send {"extra_data": {"preview_method": "latent2rgb"}}. With --preview-method taesd, H3 previews use the TAEHV decoder, which can abort the process under DynamicVRAM with aimdo memory compile error.
RAM is the real constraint: 32 GB means split with every weight cache off, 64 GB means the DiT cache plus one text-encoder cache, 96 GB+ means all of them.
One structural warning. The pack mirrors ComfyUI internals and imports private helpers from comfy-kitchen and comfy-aimdo, so a ComfyUI update can break it - or quietly drop the sliced int8 projection to a slower dequantize path. Re-run the parity tests after an update before trusting "bit-identical."
Inputs (13)
| Name | Type | Default | Description |
|---|---|---|---|
| model | MODEL | — | |
| enabled | BOOLEAN | true | — |
| second_gpu | INT | -1-1–15 | CUDA index of the second GPU; -1 = automatic. Ignored when a GPU set is connected. |
| exchange | COMBO | host | host: stage per-block exchanges through pinned RAM (safe everywhere). p2p: direct GPU-to-GPU copies (needs working P2P). |
| exchange_chunks | INT | 00–32 | Pipeline the per-block exchange in this many chunks so each card's upload overlaps its download -- PCIe is full duplex, and the serial path pays D2H + H2D where the pipeline approaches max(D2H, H2D). 0 or 1 keeps the original single-shot exchange. 8 is the measured recommendation: 1.12x on the step, 1.41x on the exchange itself. Host mode only; p2p has no two directions to overlap. Exchange is ~36% of a sparse step, so this is where the remaining headroom is. |
| sparse_attention | BOOLEAN | false | Run ComfyUI's Model Sparse Attention node inside the split instead of refusing it. Each rank runs the sparse kernel over its own attention heads, which is exact (sol-attn/sla select per head), not an approximation. Needs that node in the chain. Off: a sparse patch raises an error, as before. |
| dynamic_vram | COMBO | keep | 'off for this model' takes ComfyUI's per-model DynamicVRAM opt-out (ModelPatcherDynamic.get_non_dynamic_delegate) for the DiT only -- every other model in the process keeps DynamicVRAM. The split already works around aimdo in several places, and device tensors it allocates cannot be reused across steps under it, so vram_block_cache REQUIRES this. Costs a second load of the checkpoint, once per process. |
| vram_block_cache | BOOLEAN | false | Make VRAM the FIRST tier of the weight cache: as many leading blocks as fit live on the cards, and weight_cache holds the rest in pinned host RAM. A block on the cards is NOT pinned in RAM as well, so the host cache shrinks by that much. Sized at run time from free VRAM, so it follows resolution and clip length. Released when sampling ends -- nothing else can free it. Turn weight_cache on too, or the overflow blocks stream from pageable memory. |
| vram_reserve_gb | FLOAT | 2.00–32 | VRAM left free on each rank GPU beyond the step working set, when vram_block_cache is on. ComfyUI can see these allocations but can never reclaim them, so leaving it nothing makes ComfyUI start unloading its own models. Raise it if a later node (the VAE split worker needs ~5.2 GiB) runs short. Default comes from H3MS_VRAM_SAFETY_GIB. |
| sparse_vsa | BOOLEAN | false | Allow the 'vsa' method inside the split (needs sparse_attention on, and FastH3 weights carrying to_gate_compress). Off: a 'vsa' patch raises with a reason -- this is the rollback switch; sol-attn and sla are unaffected. Costs ~322 MB VRAM per GPU for the coarse-gate buffer. |
| weight_cache | BOOLEAN | true | Keep the block weights in pinned RAM for the life of the ComfyUI process (~18.5 GB for MiniMax H3 int8). Survives prompts and model reloads; ComfyUI never evicts it. |
| cache_ram_reserve_gb | FLOAT | 0.00–512 | Only fill the weight cache while at least this much system RAM would stay available. 0 = use RAM as needed. |
| gpusopt | H3MS_GPUS | GPU selection from H3 MS GPU Set. Without it: every visible GPU (or second_gpu). |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| MODEL | MODEL | — |