H3 Frozen Video Cache

The other two nodes in this pack fix MiniMax H3's audio - but each refinement step costs nearly a full model forward. On the author's RTX 5090, an audio-only refinement step took 20.7s against 23.0s for a full joint step: all ~37k frozen video tokens still run through all 50 blocks, and the output is thrown away. This node computes those frozen rows once, then lets later audio-only steps reuse them.

How it works

On the first refinement step it runs the model normally while recording each block's post-rope attention K/V (or post-norm hidden states) for the frozen rows - text, conditioning, video. On every later step it computes only the audio rows, about 1% of the sequence, attending against the cached K/V. Two honest caveats: the first step is always full price (that's the build), and it's an approximation - between rebuilds the frozen rows stop reacting to the evolving audio. refresh_interval N reopens that feedback path, at full cost per rebuild.

Wiring it up

Wiring is deliberately simple: MODEL in, MODEL out. Place it after your LoRA/patch stack and feed the patched MODEL into H3 Audio Refine Sampler (or a stock sampler fed by H3 Audio Refine Mask). It self-gates - it only activates when video is fully frozen and audio still generating, so pass-1 sampling passes through bit-for-bit. The first refinement step logs building cache | rows=... size=... backend=...; never see that line, and the cache isn't engaging.

Measured on the author's machine (one clip, cache in system RAM at hidden/int4): 5.7x per cached step (20.7s → 3.6s), 2.8x on the refinement pass, 1.4x end-to-end, with the build paying for itself after ~1.3 steps. Not a promise - the saving is compute removal, so it shrinks where weight streaming dominates.

The knobs that matter

• cache_contents - hidden (default) stores post-norm hidden states and rebuilds K/V on the fly: ~2.7x smaller than kv, cached steps keep ~30% of the video matmul, ~3x faster. kv stores K/V directly: nearly free steps, but 2.7x the size.
• backend - auto (default) picks the first of vram/ram/disk that fits and logs why; or force one.
• precision - int4 (default, smallest and fastest to stream), fp8 (2x), bf16 (4x, exact). Deviation is small against the approximation itself (int4 ~2e-3).
• refresh_interval - 0 = build once. Raise it if you want the frozen rows to periodically re-see the audio.
• verbose - per-step diagnostics so you can see where the time goes.
• vram_margin_gb - raise it if you hit CUDA OOM; it makes ComfyUI evict more before the cache allocates.

The disk gotcha

Now the warning, and it's the reason the defaults are what they are: disk caching is off by default, and keep it that way unless you mean it. With allow_disk on, the cache writes the whole thing to your drive on every build - around 10 GB per run on the README's clip, again on every invalidation. Thirty renders in an evening is ~300 GB written; daily, ~110 TB a year against a typical consumer NVMe's 600–1200 TBW endurance - scratch data, thrown away. It raises a clear error rather than silently falling back to disk when nothing fits. If you're memory-poor, prefer hidden/int4, or just bypass the node - refinement works without a cache, it's just slower, and exact.

Sizing: at 1344×768/24fps, hidden/int4 is roughly 1 GB per second of clip, scaling linearly with resolution and frame count; kv 2.7x, fp8 2x, bf16 4x. The exact number is printed to the console before every build - that log line is authoritative.

Do you want it?

Memory tight → skip it; the uncached pass costs nothing extra and is exact. Can spare ~10 GB and run a couple of refinement steps or more → use it. Not sure → verbose and A/B against enabled off - no rewiring.

Troubleshooting

If you hit OOM, the README's first suspect isn't the cache: ComfyUI has a known CUDA driver bug in dynamic VRAM streaming (issue #15255), and the documented workaround is a pair of startup flags:

python main.py --cuda-device 0 --disable-pinned-memory

If verbose shows 0 of 50 blocks on the cached path, the block replacement never ran and the cache can't take effect.

Installing

Install via ComfyUI Manager (search ComfyUI-H3-AudioRefine) or git clone https://github.com/Adudeguyman/ComfyUI-H3-AudioRefine.git into custom_nodes, restart. No Python dependencies, but it needs a ComfyUI with native H3 support including the AV masked path (0.33.x against master).