T5 Quantization Config 🐼

What it is

A very specific, very technical node: it exposes Hugging Face's bitsandbytes quantization settings as ComfyUI widgets, so you can shrink the superprompt-v1 T5 model's memory footprint before loading it. If you don't recognize field names like bnb_4bit_quant_type or llm_int8_threshold, that's your signal this node isn't for you - the honest answer for most people is to skip it entirely and load T5 at full precision through Load T5 Model, since superprompt-v1 is small enough that quantizing it rarely buys you much.

Where it matters: if you're running on a genuinely VRAM-starved card and stacking this pack's T5 model on top of everything else you've already got loaded (a checkpoint, a CLIP, maybe a couple of the pack's other VLMs), this is how you claw some of that memory back for the T5 model specifically.

This node builds a bitsandbytes config, and it's worth knowing it's a different quantization approach from the GGUF Q4-Q8 ladder that's become the standard way to shrink diffusion transformers and their text encoders. bitsandbytes 8-bit and especially 4-bit (the NF4 style this node's bnb_4bit_* fields configure) is generally regarded as the roughest cut available - real memory savings, but the biggest quality hit among the common quantization options. Reach for load_in_8bit before load_in_4bit if you have to quantize at all.

How it works

Nothing here runs a model - it just assembles a config object matching bitsandbytes' BitsAndBytesConfig, which you then wire into Load T5 Model's optional quantizationConfig input. That node does the actual loading with these settings applied.

Inputs and outputs

• quantization_mode - none (default), load_in_8bit, or load_in_4bit. The real on/off switch; the fields below only matter once you've picked one of the two active modes.
• llm_int8_threshold - default 6. An 8-bit-mode outlier threshold; leave at default unless you know why you're changing it.
• llm_int8_skip_modules - default empty. Comma-separated module names to exclude from 8-bit quantization if a specific layer needs full precision.
• llm_int8_enable_fp32_cpu_offload - default off. Offloads overflow to CPU in fp32 rather than failing when a layer won't fit in 8-bit on GPU.
• llm_int8_has_fp16_weight - default off, an 8-bit-mode detail about the source weight format.
• bnb_4bit_compute_dtype - default float32, the dtype used for compute during 4-bit inference.
• bnb_4bit_quant_type - default fp4; nf4 is the other common choice if you're deliberately choosing 4-bit.
• bnb_4bit_use_double_quant - default off. A further compression trick that quantizes the quantization constants themselves for extra savings.
• bnb_4bit_quant_storage - default uint8, the storage dtype for the packed 4-bit weights.

Output: QuantizationConfig (QUANTIZATIONCONFIG), which only wires into one place - Load T5 Model's optional input.

Installing it

Part of the same pack:

cd ComfyUI/custom_nodes
git clone https://github.com/zhongpei/Comfyui_image2prompt

Or search "Comfyui_image2prompt" in ComfyUI Manager. (The README's clone snippet is typo'd to a hyphenated URL - use the underscored one above.) Restart ComfyUI. This node needs bitsandbytes installed as a Python dependency for the config to actually be usable at load time - if it's missing, you'll see the failure when Load T5 Model tries to use the config, not when this node runs.

Common issues

Reaching for this when you don't need to. superprompt-v1 is small. Quantizing it is solving a VRAM problem you probably don't have, at the cost of the same quality tax any 4-bit quantization carries. Try the default (no quantization config wired in) first.

Setting fields without picking a mode. If quantization_mode stays none, every other field here is inert - the config you build won't apply anything. Set the mode first.

4-bit over 8-bit for no reason. load_in_4bit is the more aggressive, lossier option. Start with load_in_8bit if you need any quantization at all, and only drop to 4-bit if 8-bit still doesn't fit.