TryOff Quantizer [ComfyUI-Flux-TryOff]

A 12B Flux transformer plus T5 and a VAE is a lot of model to cram onto a consumer card. TryOff Quantizer is the escape hatch: it's a two-output config generator that tells the pack's loaders to quantize with bitsandbytes instead of loading everything in bf16. It's a tiny node, but it's the difference between "runs on my 4080" and "CUDA out of memory on node load."

What it is

One required input, three choices:

• None - full precision (well, bf16). The default, and the README is clear you can run this way if you have the VRAM.
• 8Bit - the recommended middle ground. The README's flagship workflow is literally named 8BitQuantized.json.
• 4Bit - the "make it fit or bust" option for lower-end GPUs. Slower, more risk of quality drift, but the model actually loads.

It produces two outputs:

• transformers_config - a bitsandbytes load_in_8bit/load_in_4bit config for the Hugging Face transformers side (the text encoders).
• diffusers_config - the matching config for the diffusers side (the transformer fine-tune and VAE).

Both are typed outputs (transformers_config and diffusers_config), and they're designed to be wired in as a matched pair - transformers_config into the TryOff Model Loader, diffusers_config into the FluxFill loaders. Mixing them up is the one way to get this wrong, and it fails as a mysterious type error.

Why you'd bother

This is the same play as GGUF in the broader Flux ecosystem - quantized weights to trade a little quality for a huge VRAM win. The author's framing: the quantized versions "will work on lower end GPUs," with the caveat that multi-GPU runs are untested. The sample 8BitQuantized.json workflow runs the whole try-off graph at 8-bit and drops the step count to 20 (from the run node's 50-step default) to keep runtime tolerable.

The honest take: try None first if your card has the headroom - cleaner output, and bf16 Flux is what everyone benchmarked. Reach for 8-bit the moment OOM errors appear (they will, if you're on 16GB with the legacy loader's full VAE). 4-bit is the last resort; expect softer textures and longer per-step time, which partially defeats the purpose.

The bitsandbytes catch

Here's the trap: requirements.txt lists diffusers, transformers, accelerate, and numpy - not bitsandbytes. The quantizer only works if bitsandbytes is already in your Python environment (it ships with the ComfyUI portable install on Windows, and is usually present in Linux envs, but "usually" is not "guaranteed"). If you flip to 8Bit/4Bit and get an import or "bitsandbytes was not found" error, that's why:

pip install bitsandbytes

On Windows, use the pip install of the correct wheel for your CUDA version. Also worth knowing: the 8-bit/4-bit path skips the .to(device) step and lets the loaders use device_map/CPU offload instead, so the quantized workflows rely on offloading more heavily - which is exactly how they squeeze onto smaller cards.

Install

The pack itself is one line via ComfyUI Manager (search "ComfyUI-Flux-TryOff") or git clone https://github.com/asutermo/ComfyUI-Flux-TryOff into custom_nodes, then restart. There's nothing extra to download for this node - it's pure config plumbing. The models it helps you load download automatically.