Nodes/ComfyUI-FunPack/FunPack Diffusion Model Loader
ComfyUI Node

FunPack Diffusion Model Loader

Fp8, GGUF and an attention backend in one node

By digital-garbage·Created about a year ago·Updated about 20 hours ago· 18
FunPack Diffusion Model Loader
    • MODEL
    • status
    model_name
    weight_dtypedefault
    compute_dtypedefault
    attentiondefault
    fp16_accumulationfalse
    slafalse
    sla_sparsity0.90
    sla_block_size64
    sla_protect_audiotrue
    sla_min_seq_len8192
    sla_dense_last_steps0

    ComfyUI's built-in UNETLoader gets you a model in and out, but that's about it. The FunPack Diffusion Model Loader is what happens when someone got tired of stacking a GGUF pack, an fp8 loader and an attention override into the same graph - it's a drop-in loader that puts weight dtype, compute dtype, an attention backend and even MiniMax H3's sparse-attention speedup into one node, and then prints a status string telling you exactly what it decided. It ships inside ComfyUI-FunPack, DigitalGarbage's hobbyist UI/utility pack, and for regular ComfyUI use it's the pack's most useful node that doesn't require the Cutting Room editor at all.

    What it actually does

    The required inputs read like a list of things you usually patch in with three separate packs:

    • model_name - picked from ComfyUI/models/diffusion_models. The neat part: .gguf files are listed right alongside .safetensors ones, and the loader reads GGUF itself, so you don't need ComfyUI-GGUF installed for it to work.
    • weight_dtype - how weights sit in VRAM: default, fp8_e4m3fn, fp8_e4m3fn_fast, fp8_e5m2, fp16, bf16, fp32. The _fast variant also runs fp8 matmuls, which is quicker on Ada and newer at a hair of precision cost; plain fp8_e4m3fn roughly halves memory with barely visible loss. This is the same fp8 story as everywhere else in ComfyUI - on a 12–16GB card it's usually the first knob you touch.
    • compute_dtype - what the model computes in, separately from how weights are stored. Set it and the node stops casting every layer, so an fp8 model can compute in bf16.
    • attention - the attention backend for this model only: default, pytorch, sage, split, sub_quad, whatever's installed on your machine (only actually-available backends appear in the list). default just uses what ComfyUI was launched with. This is the same mechanism the --use-sage-attention / --use-flash-attention flags control, but per-node instead of per-launch.

    Optional inputs worth knowing: fp16_accumulation (faster fp16 matmuls on recent NVIDIA cards; silently ignored if your torch build doesn't support it) and the whole sla_* family, which is the interesting part.

    The SLA knob is why this node exists

    sla is block-sparse attention built for MiniMax H3's long packed sequence - the inference path lightx2v's SLA turbo LoRA was distilled against, which is why that LoRA gives no speedup on its own. Turn it on and you get roughly 3.7x attention throughput at 768p/15s, with the chosen attention backend running alongside it as the dense fall-through for the text refiner and masked calls. It's skipped, with the reason in the status string, on anything that isn't H3 or on a machine without Triton.

    The sub-knobs are all validated at their defaults, and the tooltips are unusually honest about the traps. sla_sparsity defaults to 0.90; break-even is around 0.60, so lowering it below that makes the kernel slower than dense - a loss, not a safe fallback. sla_block_size of 128 forces 1.6 seconds of speech down one attention pattern, so leave it at 64 unless the audio doesn't matter. And the classic "soundtrack is garbage on H3" complaint comes from 4-step distillation, not sparsity - use 6 steps instead of fiddling with this node. Keep sla_protect_audio on; it costs ~7% but stops top-k from dropping the audio prefix entirely.

    Outputs

    You get a MODEL that wires straight into a KSampler, plus a status string - newline-joined notes about the GGUF backend, dtype choices, and whether SLA installed. It's a genuinely useful diagnostic that most loaders don't bother with.

    Install

    FunPack is on the Comfy Registry and the regular routes all work:

    # ComfyUI Manager: search "ComfyUI-FunPack" and Install
    cd ComfyUI/custom_nodes
    git clone https://github.com/digital-garbage/ComfyUI-FunPack
    

    Then pip install -r requirements.txt - that pulls gguf, insightface, opencv-python-headless and the rest, with a baseline of transformers >= 5.0.0. The pack ships no models; whatever you load has to be in your diffusion_models folder already.

    Gotchas

    One thing the loader is quietly proud of: it decides GGUF by file content, not extension, so a GGUF renamed to .safetensors still loads - and it'll tell you in the status string. On the flip side, loading MiniMax H3 at fp16 or fp8_e4m3fn_fast is measured to corrupt the audio first, so keep H3 at bf16 weights. And since FunPack v4, the author updates the Cutting Room editor first and only tests nodes against it - this loader works fine in stock ComfyUI, but treat the whole pack as a solo hobby project, bug reports welcome on GitHub issues rather than expected to be fixed instantly.

    CategoryFunPack/Loaders

    Inputs (11)

    NameTypeDefaultDescription
    model_nameCOMBOThe diffusion model file, from ComfyUI/models/diffusion_models. .gguf files are listed too (see the status output for which GGUF backend loaded it).
    weight_dtypeCOMBOdefaultHow weights are stored in VRAM. fp8_e4m3fn roughly halves the model's memory; _fast additionally runs fp8 matmuls, which is faster on Ada and newer and slightly less precise. 'default' keeps whatever the file already is.
    compute_dtypeCOMBOdefaultThe dtype the model computes in. Setting this stops per-weight casting, so an fp8 model computes in bf16 without re-casting every layer.
    attentionCOMBOdefaultAttention backend for THIS model. 'default' uses whatever ComfyUI was launched with. Only backends installed on this machine are listed. Composes with `sla`: sparse attention handles H3's long packed sequence, this backend handles everything else.
    fp16_accumulationoptBOOLEANfalseAccumulate fp16 matmuls in fp16. Faster on recent NVIDIA cards; needs a torch build that supports it, ignored otherwise.
    slaoptBOOLEANfalseBlock-sparse attention for MiniMax H3 — the inference path lightx2v's SLA turbo LoRA was distilled against, which is why that LoRA gives no speedup on its own. Roughly 3.7x the attention throughput at 768p/15s. It runs ALONGSIDE the attention backend above rather than replacing it: SLA takes H3's long packed self-attention, the chosen backend (sage3, int8, flash…) takes the text refiner, masked calls and any trailing dense steps. Skipped, with the reason, on anything that is not MiniMax H3 or on a machine without Triton.
    sla_sparsityoptFLOAT0.900–0.95Fraction of key blocks skipped, when attention is sla_h3. 0.90 is validated; 0.85 is lightx2v's own value and ~15% slower. Break-even is around 0.60 — below that the kernel is SLOWER than dense, so a low value is a loss, not a safe fallback. Speech artefacts on H3 come from 4-step distillation, not from sparsity: use 6 steps rather than lowering this.
    sla_block_sizeoptCOMBO64How many sequence tokens share one key selection. H3 packs audio at 80 rows per second, so a 128-row block forces 1.6s of speech down one attention pattern while the same rows are 3% of a video frame. Total attention work is identical either way — only the routing granularity changes. Use 128 only when the audio does not matter.
    sla_protect_audiooptBOOLEANtrueAlways attend the [text | cond | audio] prefix, whatever top-k picks. Audio is ~1% of the packed sequence, so plain top-k regularly drops all of it and the soundtrack degrades while the video still looks fine. Costs about 7%.
    sla_min_seq_lenoptINT81920–1000000Sequences shorter than this stay dense. Guards the short text refiner, which must never be sparsified, and low-resolution runs where block selection costs more than it saves.
    sla_dense_last_stepsoptINT00–8Run the last N sampling steps at full attention. 0 matches lightx2v; 1 was tested and did not help, for +20% time.

    Outputs (2)

    NameTypeDescription
    MODELMODEL
    statusSTRING