🚀 Flux KV Cache (GGUF)

If you run Flux-family models as GGUF with reference images - which is how a lot of people fit Flux on a 12GB card - you've probably watched the standard Flux KV Cache node quietly stop working. It caches the reference-image K/V tokens on the GPU, then something offloads the model (a VAE pass, a clip pass, memory pressure) and the cache is gone. Every denoise step recomputes those reference tokens from scratch, and the whole "cache" trick is dead. This node fixes exactly that failure mode.

It slots between your Unet Loader (GGUF) and the sampler: Unet Loader (GGUF) → Flux KV Cache (GGUF) → KSampler. The pack's own YarvixPA - Flux2 9B KV - GGUF workflow runs it with a Multi-ReferenceLatent feeding three reference images, so this is the node those multi-reference Flux2 GGUF setups are built around.

How it works

During the first denoise step, the patch it installs grabs the K/V tensors for the reference-image tokens and stores them. On later steps it reuses those cached tensors instead of recomputing them. Three details make it behave differently from the original KV Cache node:

• CPU cache by default. The cache_on_cpu toggle (default on) stores the cached K/V on system RAM, so model offload/reload cycles - which happen constantly with GGUF - can't destroy it. The trade-off is that each step pulls the tensors back from CPU, which is slower per step but still a huge net win over recomputing. Turn it off to keep the cache on the GPU if you have the headroom and know nothing will evict it.
• cleanup() is a no-op. Where the stock node clears its cache when cleanup is called (and GGUF models get cleanup called a lot), this one deliberately doesn't - the cache lives for the whole generation and is replaced when a new one starts.
• It forces re-execution. IS_CHANGED returns NaN so ComfyUI re-runs the node every prompt. Without that, ComfyUI's output caching would hand you back the old patch object with stale K/V the moment your reference images change - the kind of bug that makes you question your sanity.

It also sets default_ref_method to index_timestep_zero, which pins how the reference tokens are attended to across steps so the cached values stay valid.

Inputs and outputs

Dead simple - that's the point.

• model (required) - the MODEL output from your Unet Loader (GGUF).
• cache_on_cpu (optional, default on) - where the cache lives, as above.

Output is a MODEL that goes into your sampler. Nothing else to configure.

Install

Same pack as everything else in ComfyUI-YarvixPA: ComfyUI Manager (search "YarvixPA") or

cd ComfyUI/custom_nodes
git clone https://github.com/YarvixPA/ComfyUI-YarvixPA

then restart. No extra models to download - it's a patch node, pure logic.

When it's worth it, and when it isn't

It only does anything if your workflow has reference images (via reference_image_num_tokens in the transformer options). No references, no cache, no win - the node becomes a pass-through with a tiny overhead, so don't expect magic on plain txt2img. And it's a GGUF-specific fix: the stock Flux KV Cache node works fine with full fp8/fp16 models that don't thrash offloads. On GGUF with reference-heavy workflows, though, this is the one to reach for. If you're on a card where VRAM is tight and the standard node visibly stalls mid-generation, this is usually the cure.