LayerStream: H3 Chunking

MiniMax H3 is a 33B omni-modal video model that generates picture and stereo audio in one pass, and 15 seconds at a megapixel is a very long sequence. Ask for that on a 24 GB card and you get an out-of-memory error, or the quieter failure where activations spill into shared system memory and the render crawls for an hour.

LayerStream: H3 Chunking is the interesting answer to that, because it makes a promise almost nobody in the low-VRAM space makes: the output is bit-identical to stock ComfyUI. Not "visually the same". The author compared decoded frames against a stock run and got PSNR = infinity - 124 frames at six steps, then the whole 362-frame, 15-second clip at two and six.

The wall is activations, not weights

ComfyUI streams weights layer by layer through its Dynamic VRAM path, so a 20-25 GB model file doesn't need 20-25 GB resident. What it does not shrink is activation memory - the qkv buffers, the MLP expansion, the fp32 norm and output-head copies, the residual stream. That grows with token count, and token count is frames × resolution. H3 doesn't fail because the model is big. It fails because the clip is long.

How the slicing works

Norms, modulation, the MLP and the projections are all token-wise - every token computes independently - so they run on slices of the sequence and accumulate into the residual stream in place. Same math, smaller working set.

Attention isn't token-wise: every query needs every key and value. So it runs in two passes. Pass one keeps nothing but K and V for all tokens. Pass two recomputes the queries per slice, attends that slice against the full K/V, projects, and adds into the residual. You pay for the query projection twice; you don't pay to hold a giant attention buffer.

The fp32 output head runs stock first - exact - and only slices when it runs out of memory, which is why the exactness claim holds.

Mechanically, the pack patches DiTBlock.forward and FinalLayer.forward at import time; the node clones your model and stashes the chunk sizes in its transformer_options, so the patched code takes the stock path whenever the node isn't wired in. That's why bypassing it (Ctrl+B) is a legitimate A/B test.

The two numbers you set

model in, model out. Chain it after your LoRA loader and before the sampler or guider, same as any model patch.

• mlp_chunk_tokens (default 8192, step 256) - tokens per MLP slice. Smaller means lower peak VRAM and slightly more time.
• attn_chunk_tokens (default 8192, 0 allowed) - query tokens per attention slice. Setting it to 0 turns the two-pass attention off entirely and gives you stock attention, so only the MLP half gets chunked. That's your dial-down step if you want a small saving without the attention rewrite.

Author's rule, and it's the right one: use the largest chunk that fits. Chunking costs time, mostly because streamed layer weights get fetched again for every slice.

Install

It's part of osiworx-pack, so install the pack, not the node. No pip dependencies - the requirements file is empty on purpose; it uses the torch and einops ComfyUI already ships.

cd ComfyUI/custom_nodes
git clone https://github.com/osi1880vr/osiworx-pack

Or search "osiworx-pack" in ComfyUI Manager. Restart and look under osiworx/layer. You need a ComfyUI with actual H3 support (comfy.ldm.minimax); developed against 0.39.0.

For a small VRAM budget, also launch ComfyUI with:

python main.py --vram-headroom 10 --disable-pinned-memory

The formula is N = total VRAM − the budget you want. Without it, Dynamic VRAM fills the card with weights and pushes activations into shared system memory - exactly the slow path you were trying to escape.

Where people get burned

Try the launch flag without the node first. Honest disclosure from the author: on his 4090, --vram-headroom 10 --disable-pinned-memory alone finished the 15-second clip. The node's added value is a lower activation peak, demonstrated under a hard 8 GB cap on PyTorch's own allocations. That is not the same as proving an 8 GB card works.

There's a floor around 4.2 GiB. A hard 4 GB cap failed, because K and V for the whole sequence stay resident - about 1.5 GiB each in his test. Going lower needs K/V offload, which this design doesn't do. Turning the chunk down to 256 won't get you there.

Other attention backends are untested. The bit-identical result used ComfyUI's default attention. One user got a big speedup adding a "Model Attention Backend" node set to Comfy Kitchen before the LayerStream node - one run, one machine, treat it as a hint and check your output.

If the node isn't in the menu, your ComfyUI is missing comfy.ldm.minimax or has moved it. The pack is import-safe: the module fails, logs a warning, the rest still loads. Grep your startup log for osiworx-pack.

Don't judge it at 1-2 steps. H3 clips look broken there with or without chunking. The turbo LoRA wants about six.