Nunchaku Omini Kontext Patch

FLUX.1-Kontext is a 12B model. At fp16 that's ~24GB of VRAM just for the transformer, which rules it out for most people's cards. The usual answer is GGUF or fp8 - smaller, still decent. The faster, smaller answer is Nunchaku: MIT Han Lab's SVDQuant engine that runs the whole transformer at 4-bit weights and activations, dropping Flux to around 6.6GB with a 2–3× speedup over fp16 on top of the memory savings. That's the difference between "nice demo" and "actually my daily driver."

NunchakuOminiKontextPatch is the Omini Kontext version of that: same job as OminiKontextModelPatch, but written for the nunchaku model wrapper instead of the stock Flux one. You feed it the nunchaku-quantized Kontext MODEL, it swaps in the reference-aware forward pass, and - the part that matters - it also composes your Omini Kontext LoRA into the int4 model. That's not optional plumbing: nunchaku can't apply a LoRA the way stock ComfyUI does, it has to bake the LoRA weights into the quantized kernels. This node does that for you, so a workflow that would otherwise need extra LoRA-conversion steps just works.

What you need before it works

This node assumes you already have the nunchaku stack, and the pack README targets ComfyUI-nunchaku v0.3.4 era. You need:

1. The ComfyUI-nunchaku extension installed (Manager search "Nunchaku", or git clone https://github.com/nunchaku-ai/ComfyUI-nunchaku into custom_nodes).
2. The 4-bit Flux Kontext weights from the nunchaku-ai HuggingFace org - int4 for most cards, fp4 (NVFP4) for RTX 50-series. Loading the wrong precision for your GPU is a classic first-run failure.
3. Your Omini Kontext LoRA of choice, loaded at 0.5–0.7 strength with CFG 1.5.

Then load the int4 Kontext via the nunchaku loader node, patch it with NunchakuOminiKontextPatch, and pair it with OminiKontextConditioning using the delta values that match your LoRA - [0,0,96] for the character/product models, [1,0,0] for spatial.

Nunchaku gotchas worth knowing

Where people get burned, roughly in order of frequency:

• Wrong quantization for your GPU. RTX 50-series wants fp4; older cards want int4. Mix them up and you get broken output or outright errors. There was real Turing-era instability on 20-series cards with early nunchaku - v0.2.0 added 20-series compatibility, so if you're on a 20-series card, don't run an ancient build.
• It's picky about GPU support in general. Nunchaku's kernels are tuned for specific architectures. If your card isn't on the supported list, the install itself fights you, and this pack's patch can't fix that.
• LoRA must be nunchaku-compatible. The node composes the Omini LoRA for you, but a standard ComfyUI LoRA that hasn't been converted for nunchaku won't load through the usual path. Check the nunchaku docs on LoRA conversion if the load fails.

Install

Same as the rest of the pack - Manager, search Omini Kontext, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Saquib764/omini-kontext

Restart, then verify nunchaku itself imports (it's imported lazily, so a missing install won't error at startup - it'll only fail when you run the workflow).

Who this is for

This is the node that makes Omini Kontext realistic on a mid-range card, and honestly it's the version I'd reach for even with VRAM to spare, because int4 nunchaku Flux is fast. But set expectations: this whole pack is a 2025 research project, and Nunchaku support was an add-on bolted onto an already-thin codebase. When it works it's great; when it breaks, it breaks in the nunchaku layer, and you'll want the nunchaku repo's issues more than this one's.