Bernini-R Block Swap Args

You've lowered the context window, you've offloaded the text encoder, you've tried every attention backend. The 14B model still doesn't fit. BerniniR_BlockSwapArgs is the big hammer: it moves transformer blocks between GPU and CPU mid-sampling, so only a window of the model lives on the card at a time. It won't make Bernini-R fast - nothing does - but it's how you run it on VRAM that has no business running it.

How it works

The pack keeps a ring buffer of GPU slots holding a window of transformer blocks; the rest sit in a CPU "home pool." As sampling walks through the model, pre_forward swaps blocks in and out via a cursor-based ring buffer (no free-list bookkeeping, which is one of those implementation details that just means it's efficient). It also handles fp8-quantized weights transparently through its SlotEntry wrapper, so you can mix regular and quantized checkpoints. Two loading modes:

• Full - all weights pre-loaded into CPU RAM at startup. Zero disk I/O during sampling, at the cost of ~2× model size in RAM.
• Streaming (default) - blocks load from disk on demand. Much lower peak RAM (~1.05×), but the disk read can cause brief latency when the window slides to a new block group.

The inputs that matter

• block_to_swap - the master switch. Number of blocks kept in RAM; 0 disables block swap entirely. The 14B model has 40 blocks, the 1.3B has 30 - start by swapping a third to a half and work from there. If it's still OOM, swap more.
• prefetch - uses a CUDA stream to prefetch upcoming blocks during compute, hiding latency. Keep it on.
• prefetch_count - how many blocks ahead to prefetch. Higher hides more latency, costs more RAM.
• pin_memory - pins the CPU copies for truly async transfer. Speeds H2D copies, increases host RAM. Worth it if you have RAM to spare.
• loading_mode - Full vs Streaming, as above. Streaming is the sane default for most people.
• disk_workers - disk prefetch threads (Streaming only). 4 is the default and good on NVMe; the tooltip suggests 2 is safer on SATA.

Output and wiring

One output: block_swap_args (BERNINI_BLOCKSWAP) → the block_swap_args input on BerniniR_KSampler or BerniniR_DualExpertSampler. Leave it disconnected to disable. The pack's own example workflow connects one, so this isn't exotic - it's part of the intended stack for heavy edits.

The honest read

Block swap is a compatibility feature, not a performance feature. Swapping weights between RAM and VRAM is slower than just having the model on the card - the point is fitting, not faster. Combine it with the rest of the VRAM strategy this pack layers: context windows for temporal memory, lazy loading so weights never sit in RAM until needed, CLIP offload, VAE chunking. Each layer is independent, and you stack them until your card stops complaining.

Where people get burned: leaving block_to_swap at 0 and wondering why nothing changed. It defaults to disabled. And if you see periodic stutters mid-run on Streaming mode, that's the disk I/O the README warns about - raise prefetch_count or switch to Full if you've got the RAM. If you're on a 12GB card with the 14B model, honestly, the 1.3B checkpoint exists and it stays close on style transfer and local edits - block swap is for when you need the 14B, not for when it would merely be nice.