FunPack Diffusion Model Loader
Fp8, GGUF and an attention backend in one node
- model
- status
ComfyUI's built-in UNETLoader gets you a model in and out, but that's about it. The FunPack Diffusion Model Loader is what happens when someone got tired of stacking a GGUF pack, an fp8 loader and an attention override into the same graph - it's a drop-in loader that puts weight dtype, compute dtype, an attention backend and even MiniMax H3's sparse-attention speedup into one node, and then prints a status string telling you exactly what it decided. It ships inside ComfyUI-FunPack, DigitalGarbage's hobbyist UI/utility pack, and for regular ComfyUI use it's the pack's most useful node that doesn't require the Cutting Room editor at all.
What it actually does
The required inputs read like a list of things you usually patch in with three separate packs:
- model_name - picked from
ComfyUI/models/diffusion_models. The neat part:.gguffiles are listed right alongside.safetensorsones, and the loader reads GGUF itself, so you don't need ComfyUI-GGUF installed for it to work. - weight_dtype - how weights sit in VRAM:
default,fp8_e4m3fn,fp8_e4m3fn_fast,fp8_e5m2,fp16,bf16,fp32. The_fastvariant also runs fp8 matmuls, which is quicker on Ada and newer at a hair of precision cost; plainfp8_e4m3fnroughly halves memory with barely visible loss. This is the same fp8 story as everywhere else in ComfyUI - on a 12–16GB card it's usually the first knob you touch. - compute_dtype - what the model computes in, separately from how weights are stored. Set it and the node stops casting every layer, so an fp8 model can compute in bf16.
- attention - the attention backend for this model only:
default,pytorch,sage,split,sub_quad, whatever's installed on your machine (only actually-available backends appear in the list).defaultjust uses what ComfyUI was launched with. This is the same mechanism the--use-sage-attention/--use-flash-attentionflags control, but per-node instead of per-launch.
Optional inputs worth knowing: fp16_accumulation (faster fp16 matmuls on recent NVIDIA cards; silently ignored if your torch build doesn't support it) and the whole sla_* family, which is the interesting part.
The SLA knob is why this node exists
sla is block-sparse attention built for MiniMax H3's long packed sequence - the inference path lightx2v's SLA turbo LoRA was distilled against, which is why that LoRA gives no speedup on its own. Turn it on and you get roughly 3.7x attention throughput at 768p/15s, with the chosen attention backend running alongside it as the dense fall-through for the text refiner and masked calls. It's skipped, with the reason in the status string, on anything that isn't H3 or on a machine without Triton.
The sub-knobs are all validated at their defaults, and the tooltips are unusually honest about the traps. sla_sparsity defaults to 0.90; break-even is around 0.60, so lowering it below that makes the kernel slower than dense - a loss, not a safe fallback. sla_block_size of 128 forces 1.6 seconds of speech down one attention pattern, so leave it at 64 unless the audio doesn't matter. And the classic "soundtrack is garbage on H3" complaint comes from 4-step distillation, not sparsity - use 6 steps instead of fiddling with this node. Keep sla_protect_audio on; it costs ~7% but stops top-k from dropping the audio prefix entirely.
Outputs
You get a MODEL that wires straight into a KSampler, plus a status string - newline-joined notes about the GGUF backend, dtype choices, and whether SLA installed. It's a genuinely useful diagnostic that most loaders don't bother with.
Install
FunPack is on the Comfy Registry and the regular routes all work:
# ComfyUI Manager: search "ComfyUI-FunPack" and Install
cd ComfyUI/custom_nodes
git clone https://github.com/digital-garbage/ComfyUI-FunPack
Then pip install -r requirements.txt - that pulls gguf, insightface, opencv-python-headless and the rest, with a baseline of transformers >= 5.0.0. The pack ships no models; whatever you load has to be in your diffusion_models folder already.
Gotchas
One thing the loader is quietly proud of: it decides GGUF by file content, not extension, so a GGUF renamed to .safetensors still loads - and it'll tell you in the status string. On the flip side, loading MiniMax H3 at fp16 or fp8_e4m3fn_fast is measured to corrupt the audio first, so keep H3 at bf16 weights. And since FunPack v4, the author updates the Cutting Room editor first and only tests nodes against it - this loader works fine in stock ComfyUI, but treat the whole pack as a solo hobby project, bug reports welcome on GitHub issues rather than expected to be fixed instantly.
Inputs (19)
| Name | Type | Default | Description |
|---|---|---|---|
| model_name | COMBO | 0 options: | |
| weight_dtype | COMBO | default | How the weights are stored. 'default' keeps whatever the file already is. |
| compute_dtype | COMBO | default | What the maths runs in, independently of storage. |
| attention | COMBO | default | Attention backend. Only what this machine can run is listed; 'default' keeps ComfyUI's launch choice. |
| fp16_accumulationopt | BOOLEAN | false | Faster fp16 matmuls where the torch build supports it. |
| slaopt | BOOLEAN | false | Block-sparse attention for MiniMax H3 -- the inference path lightx2v's SLA turbo LoRA was distilled against, which is why that LoRA gives no speedup on its own. Roughly 3.7x the attention throughput at 768p/15s. Runs ALONGSIDE the attention backend above rather than replacing it: SLA takes H3's long packed self-attention, the chosen backend takes the text refiner, masked calls and any trailing dense steps. Skipped, with the reason, on anything that is not MiniMax H3 or on a machine without Triton. |
| sla_sparsityopt | FLOAT | 0.900–0.95 | Fraction of key blocks skipped while SLA is on (of video cubes for method vsa). 0.90 is validated; 0.85 is lightx2v's own value and ~15% slower. Break-even is around 0.60 -- below that the kernel is SLOWER than dense, so a low value is a loss, not a safe fallback. |
| sla_block_sizeopt | COMBO | 64 | How many sequence tokens share one key selection. H3 packs audio at 80 rows per second, so a 128-row block forces 1.6s of speech down one attention pattern while the same rows are 3% of a video frame. Use 128 only when the audio does not matter; 32 routes finer still for about the same time. Triton engine only. |
| sla_protect_audioopt | BOOLEAN | true | Always attend the prompt's words and every audio stream, whatever top-k picks. Audio is ~1% of the packed sequence, so plain top-k regularly drops all of it and the soundtrack degrades while the video still looks fine. Reference images stay sparse: see sla_references. |
| sla_min_seq_lenopt | INT | 81920–1000000 | Sequences shorter than this stay dense. Guards the short text refiner, which must never be sparsified, and low-resolution runs where block selection costs more than it saves. |
| sla_dense_last_stepsopt | INT | 00–8 | Run the last N sampling steps at full attention. 0 matches lightx2v; 1 was tested and did not help, for +20% time. |
| int8_convrotopt | BOOLEAN | false | MiniMax H3 from an ordinary (bf16/fp16) file: store the attention and MLP weights as int8 (rotated first, so no lone outlier spoils the rounding), the way Comfy-Org's own int8_convrot files are. About half the memory and int8 matmuls on GPUs that have them; slightly different pictures. Does nothing to a file that is already quantized or is not H3 (the status says so), and is skipped when a weight dtype is forced above. The memory saving always holds; the speed-up is lost on layers a LoRA is applied to and when compute dtype is forced. |
| sla_engineopt | COMBO | comfy_kitchen | What runs the sparse calls. comfy_kitchen: ComfyUI's compiled int8 sol_attn (needs comfy-kitchen 0.2.32+; the Triton kernel runs instead where it is missing, and the status says so). triton: FunPack's own kernel, the only one with block size and stabilize motion. |
| sla_dense_stepsopt | STRING | 1 | Steps at full attention, counted from 1: '1' is the first step, '1-2' the first two, '1,6' the first and sixth; blank or '0', none. The first steps set the layout and how closely the prompt is followed, so the first is exact by default. Adds to dense last steps. The log names the steps that ran this way. |
| sla_referencesopt | COMBO | off | Reference images, conditioning frames and the prompt's vision tokens. off: sparse like the video. light: each keeps its best 15% for every query block. heavy: all of them exact, slow with big references. On comfy_kitchen, light picks one 15% for all query blocks. |
| sla_tailopt | BOOLEAN | false | comfy_kitchen engine: the skipped blocks still count, as one averaged term per query block, instead of not at all. Off matches what the SLA turbo LoRA was trained against; untested on H3. |
| sla_stabilize_motionopt | BOOLEAN | false | Triton engine: each layer leans toward the blocks it picked last step, so near ties stop flipping and showing as a faint double exposure on fast motion. A fix for that one symptom; a little more memory. |
| sla_methodopt | COMBO | sla | How the blocks worth attending are chosen. sla: a fixed share (sla_sparsity), what SLA turbo LoRAs were trained against. sol-attn: a per-block threshold (sla_tau), no training needed; comfy_kitchen engine. vsa: video cubes, only for models trained for it (the status says when the model lacks VSA's layers); run by ComfyUI's own Model Sparse Attention, keeping 1 - sla_sparsity of the cubes. |
| sla_tauopt | FLOAT | 1.300–4 | sol-attn only: higher skips more. About 1.0 keeps ~16% of key blocks exact, 1.5 ~7%, 2.0 ~3%. |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| model | MODEL | — |
| status | STRING | — |