TurboQuant KV Patch

If the name makes you think this node quietly frees 4.5x of your VRAM, slow down. TurboQuant KV Patch is an experimental attention patch that round-trips a model's K/V tensors through TQ3, a 3-bit quantization format. The author says it straight in the README: it's for studying KV-cache compression and validating quality - it is not a persistent KV cache yet. That honesty is the whole reason to trust the claims it does make.

Why this exists

The motivation is LTX-2.3 22B needing ~32.4GB VRAM, barely squeezing onto a V100 32GB. A large slice of that is the KV cache inside the transformer attention layers - the per-token key/value tensors every layer keeps around. Compress those from FP16 (16 bits) to TQ3 (3.5 bits effective) and that chunk of memory drops by roughly 5x. If that sounds like something out of the LLM world, it is: KV-cache quantization is a well-known LLM trick, and this is someone porting the idea into ComfyUI to see how diffusion attention holds up.

How it works

The node clones your model patcher (your original model stays untouched), then registers a patch on both attn1 (self-attention) and attn2 (cross-attention). Every time attention computes new K and V, the patch:

1. quantizes them to TQ3, then immediately decompresses them back - forcing the data through the compressed memory layout so PyTorch's allocator can free the original FP16 tensors sooner,
2. measures bytes in vs. out along the way.

TQ3 itself is the interesting part: each block of 128 floats becomes 56 bytes. It L2-normalizes, runs a Fast Walsh-Hadamard Transform to decorrelate the values, applies deterministic sign flips to spread energy, scales to [-1, +1], then hits an 8-level Lloyd-Max codebook - 3 bits per value - and packs the indices down to 48 bytes plus 4 bytes of norm and 4 of scale. The stated round-trip quality target is cosine similarity > 0.97 on typical attention vectors.

The key caveat: diffusion models recompute attention every sampling step, so there's no persistent cache being reused. The benefit is peak memory during the attention computation, not a cached-away 4.5x across the whole run. Anyone hoping this is a VRAM miracle for SDXL will be disappointed; it's a memory-mechanics experiment with a measurement node attached.

Inputs and outputs

Only two inputs matter:

• model (MODEL) - any model you'd feed a sampler.
• enabled (BOOLEAN, default true) - set false and the node just passes your model through unpatched (and clears the previous stats).

Output is a single model (MODEL), the patched clone. Wire it straight into your KSampler's MODEL input. The patch only engages when the head dimension is 128 or more, which is basically every modern model.

Installing it

ComfyUI Manager should find it by searching ComfyUI-TurboQuant (or the node name). Manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/Scottcjn/ComfyUI-TurboQuant

then restart ComfyUI. The README's own install is just ln -s ~/ComfyUI-TurboQuant . inside custom_nodes/, which works fine if you cloned the repo elsewhere. Good news on dependencies: there's no requirements.txt - it's pure PyTorch, which ComfyUI already ships, so no dependency hell. You can sanity-check the core with python -m tq3_core inside the repo.

Troubleshooting and expectations

Two things bite people here. First, no visible speedup - quantization adds work; this is about memory, not speed. Second, don't read "4.5x" as total VRAM - it's 4.5x on the KV-cache portion only, and the round-trip is transient. Community reaction to KV compression in diffusion has been skeptical for good reason: with video generation, VAE decode is usually the real VRAM hog, not attention. And remember to add TurboQuant Info to the graph or you'll never see whether the patch did anything. It's a genuinely clever experiment worth a few minutes of fiddling - just don't expect it to save your OOMing workflow yet.