GGUF Accelerator (patch_on_device)

Here's a frustrating reality of GGUF + LoRA workflows: quantized models are how most of us run big video models on 12GB cards, but throw a LoRA on top and the pipeline starts dequantizing, patching, and requantizing tensors - moving patches back and forth between CPU and GPU every sampling step. It works, and it's slow, and the slowness is mostly the transfer, not the math.

IAMCCS_GGUF_accelerator is a targeted fix for exactly that. It sits between your GGUF model loader and the rest of the graph and makes ComfyUI-GGUF keep its patches on-device instead of shuffling them per step. If you run quantized video models with LoRAs, this is one of the few nodes in this pack I'd call genuinely worth having.

How it works

The core knob is patch_on_device: when on, LoRA patch tensors are applied and kept on the GPU rather than being re-loaded from CPU each step. The node's mode decides how aggressively:

• auto_oom_safe (default) - tries patch_on_device with an eager move, but if free VRAM drops below min_free_vram_mb (default 1500), it backs off; if a CUDA OOM happens while moving, it falls back to offload (when oom_fallback is on).
• manual - you drive patch_on_device and move_patches_now yourself.

move_patches_now pre-moves patch tensors to the model's load device to avoid runtime transfers - faster, but it costs VRAM. Newer versions added move_policy (all_or_nothing vs partial_small_first/partial_large_first) and leave_free_vram_mb so the pre-move respects a VRAM budget instead of risking OOM.

Outputs: the same model (patched, ready for LoRA application and sampling) and a report string telling you what it decided - which is useful in auto_oom_safe mode, because it'll tell you when it backed off.

Placement matters

README guidance is explicit: put it after the GGUF model loader and before LoRA application/sampling. It modifies the model's patch behavior; it doesn't change sampling parameters.

Install

Part of IAMCCS-nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/IAMCCS/IAMCCS-nodes.git

Or ComfyUI Manager → "IAMCCS" → install → restart. You'll also need ComfyUI-GGUF installed for the loader it patches.

Common issues

The classic failure is OOM when move_patches_now is on and you're near your VRAM ceiling - the README's suggested fixes are real: raise min_free_vram_mb to 2000–3000, or set move_patches_now false. If you're still fighting fragmentation, it recommends PyTorch allocator tuning before launch, e.g. PYTORCH_ALLOC_CONF=backend:cudaMallocAsync (set it in your environment before starting ComfyUI, not in a node). And remember the KB's rule of thumb from the GGUF world: if you're VRAM-capped with Q8 + LoRAs, dropping to Q4 often nets you more speed than any patch-movement trick, because it's the dequant-requant cycle that hurts.