A modulation layer addon for Flux that reduces model size to 8.8B parameters without significant quality loss.
A modulation layer addon for Flux that reduces model size to 8.8B parameters without significant quality loss.
ComfyUI_FluxMod acts as a plugin for Flux, enabling you to run Flux Dev and Flux Schnell on more consumer-friendly hardware. This is achieved by utilizing a modulation layer that significantly reduces the parameter count while maintaining quality.
Note: You still need a Flux Dev or Flux Schnell model to use this addon.
It's recommended to use either ComfyUI Manager or Comfy Registry.
comfy node registry-install comfyui_fluxmod
git clone https://github.com/lodestone-rock/ComfyUI_FluxMod.git
| Node | Description | Options | | ----------------------- | ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | FluxModCheckpointLoader | Primary checkpoint loading node | ā¢ ckpt_name: Flux model path<br>ā¢ guidance_name: Modulation addon path<br>ā¢ quant_mode: Quantization selection | | KSamplerMod | Modified KSampler for 8-bit quantization | ā¢ activation_casting: Switch between bf16 and fp16 | | FluxModSamplerWrapper | Sampler wrapper for 8-bit quantization | ā¢ activation_casting: Switch between bf16 and fp16 | | SkipLayerForward | Skip specific Flux layers | ā¢ skip_mmdit_layers: Which MMDiT layer to skip<br>ā¢ skip_dit_layers: Which DIT layers to skip (-1 to disable) |
ā ļø Important: When using
float8_e4m3fn
orfloat8_e5m2
quantization modes, you must use either theKSamplerMod
node instead of the regular KSampler orFluxModSamplerWrapper
forSamplerCustom
or other sampler nodes that take aSAMPLER
input. This requirement does not apply when usingbf16
mode inFluxModCheckpointLoader
or when starting ComfyUI with--fast
.
ckpt_name
guidance_name
š” Tip: Check the
examples
folder for sample workflows. Drag and drop the workflow image into ComfyUI to get started quickly.
| Mode | Recommended GPU | VRAM Usage | Recommended | | ------------- | --------------- | ---------- | ----------- | | bf16 | 24GB+ | ~19GB | ā | | float8_e4m3fn | 12-16GB | ~10GB | ā | | float8_e5m2 | 12-16GB | ~10GB | ā |
Here are some comparison examples showing the output quality between the original Flux model and FluxMod:
<details> <summary><b>Example 1: Art Studio Scene</b></summary>
Prompt: A photo of an art studio with a cabin design, there are paint splatters over much of the wooden furnishing, there are old style windows overlooking a lake outside with a slanted ceiling with large skylights letting in natural light. There is an easel with a half-finished painting. There is a paint palette that is placed on a wooden table. There are dust speckles floating in the air illuminated by the golden hour light. There is a woman standing in a colourful dress and ruby red shoes who is painting in front of the easel.
Prompt: A photo of a room with the walls made of millions of pieces of shattered glass in rainbow colours, there is light shining through the glass causing a huge dispersion of colours across the scene. The light is refracting off all the glass in the room illuminating the room with bright hues from the colored glass.
Prompt: Text made out of foil balloons saying "Hi there fellow traveller, make sure to star this GitHub! Thank you!"
Prompt: Concept art of a ornately decorated pirate ship in outer space that is floating through a nebula with spectacular blue and purple hues. There is dust that is around the ship as it sails through the cosmos, dispersing at the bow of the ship.
Prompt: A scene with a blue block and a red ball that is placed on top of the blue block. In the background there is a green door with pink walls. To the left of the door, there is a painting on the wall which is showing a scene of an ocean wave that is orange. To the right of the door, there is a portrait of a cat that has purple eyes. There is a window to the right side of the scene that is letting in light that has a shade of blue, illuminating the carpet which has a cyan hue. There is a bed on the left side of the room that has a white pillow with orange sheets. There is a bedside table with an orange lamp and a phone that is placed on top of it. To the right side of the beside table, there is a green bin with a red recycling logo on it.
Need help? Join our Discord community for support and discussions.
Pull requests are welcome! Feel free to contribute to the project by:
Found a bug? Have a suggestion? Please create a GitHub issue with: