ComfyUI Node
Quantization Config Node
A ComfyUI node in LLM with 9 inputs and 1 output.
Quantization Config Node
- QuantizationConfig
◄quantization_modenone►
◄llm_int8_threshold6.00►
◄llm_int8_skip_modules►
◄llm_int8_enable_fp32_cpu_offloadfalse►
◄llm_int8_has_fp16_weightfalse►
◄bnb_4bit_compute_dtypefloat32►
◄bnb_4bit_quant_typefp4►
◄bnb_4bit_use_double_quantfalse►
◄bnb_4bit_quant_storageuint8►
CategoryLLM
Inputs (9)
| Name | Type | Default | Description |
|---|---|---|---|
| quantization_mode | COMBO | none | 3 options: none, load_in_8bit, load_in_4bit |
| llm_int8_threshold | FLOAT | 6.00 | — |
| llm_int8_skip_modules | STRING | — | |
| llm_int8_enable_fp32_cpu_offload | BOOLEAN | false | — |
| llm_int8_has_fp16_weight | BOOLEAN | false | — |
| bnb_4bit_compute_dtype | STRING | float32 | — |
| bnb_4bit_quant_type | STRING | fp4 | — |
| bnb_4bit_use_double_quant | BOOLEAN | false | — |
| bnb_4bit_quant_storage | STRING | uint8 | — |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| QuantizationConfig | QUANTIZATIONCONFIG | — |