Nodes/FunUI/FunUI Diffusion Model Loader
ComfyUI Node

FunUI Diffusion Model Loader

The precision knobs, the int8 trick, and the sparse-attention switch that actually earns its keep

By digital-garbage·Created about a year ago·Updated about 10 hours ago· 21
FunUI Diffusion Model Loader
    • model
    • status
    ◄model_name▾►
    ◄weight_dtypedefault►
    ◄compute_dtypedefault►
    ◄attentiondefault►
    ◄fp16_accumulationfalse►
    ◄slafalse►
    ◄sla_sparsity0.90►
    ◄sla_block_size64►
    ◄sla_protect_audiotrue►
    ◄sla_min_seq_len8192►
    ◄sla_dense_last_steps0►
    ◄int8_convrotfalse►
    ◄sla_enginecomfy_kitchen►
    ◄sla_dense_steps1►
    ◄sla_referencesoff►
    ◄sla_tailfalse►
    ◄sla_stabilize_motionfalse►
    ◄sla_methodsla►
    ◄sla_tau1.30►
    ◄sla_sparsity_start0.00►

    Most "model loader" nodes are a file picker with a job. This one has a file picker and then twenty-odd inputs, most of which change what the loaded model is rather than how it's used - which is exactly why they're widgets on the node and not preferences somewhere: nobody downstream can supply them, and a saved workflow has to record them.

    If you're on H3 and just want to generate, you can set the file and ignore almost all of it. But two or three of these knobs are the difference between "it runs" and "it runs at a usable speed", so here's the whole thing sorted by how much you should care.

    The four you actually touch

    model_name - the diffusion model file. .gguf files are listed too, even when you have no GGUF backend installed, so a saved pick stays valid and the loader can tell you what's missing.

    weight_dtype - how the weights are stored. default keeps whatever the file already is, which for most 2026 releases is already the sane answer. Forcing fp8_e4m3fn is how you take a bf16 file down to half the VRAM.

    compute_dtype - what the maths runs in, independently of storage. Mixing the two is the point: fp8 stored, bf16 computed is a normal, working combination. Leave both on default until you have a reason.

    attention - the backend. The list is built from ComfyUI's own registry, so it offers only what this machine can run: sage where SageAttention is installed, flash where flash-attn is. default keeps whatever ComfyUI picked from its launch flags. Don't touch it to "tune" anything unless you're comparing on a fixed seed.

    Also worth a mention: fp16_accumulation, which uses faster fp16 matmuls where your torch build supports them. Free speedup, occasionally numerically unhappy - keep it if your output doesn't change.

    int8_convrot: the interesting one

    Take an H3 file that's still bf16/fp16, flip int8_convrot on, and the loader stores the attention and MLP weights as int8 - rotated first so no single outlier wrecks the rounding, the same scheme as Comfy-Org's own int8 files. You get roughly half the memory and int8 matmuls on GPUs that have them. Pictures come out slightly different; that's the trade.

    It does nothing to a file that's already quantized, and nothing to a non-H3 model - the status output will say so rather than silently no-op. It's also skipped if you forced a weight dtype above it. The memory saving always lands; the speed-up doesn't, on layers that have a LoRA on them or when you force compute dtype.

    SLA: block-sparse attention, and why it matters

    SLA is the one feature here you'd read a page for. It's a block-sparse attention path for H3 - and the reason it matters is a small piece of model lore: the popular "SLA turbo" LoRA for H3 was distilled against this inference path, so the LoRA gives you no speedup on its own. Turn SLA on and the LoRA's distillation finally has something to act on. The author's figure is roughly 3.7x attention throughput at 768p/15s.

    It runs alongside the attention backend you picked rather than replacing it: SLA takes H3's long packed self-attention, the backend keeps the text refiner and masked calls. On anything that isn't H3, or a machine without Triton, it's skipped and the status says why.

    The sub-knobs, short version:

    • sla_sparsity - 0.9 is the validated number; 0.85 is lightx2v's own and is about 15% slower. Below roughly 0.6 the kernel is slower than dense attention, so a low value is a loss dressed up as a safe fallback.
    • sla_protect_audio - leave it on. Audio is about 1% of H3's packed sequence, so plain top-k selection will happily drop all of it and your soundtrack degrades while the video still looks fine.
    • sla_dense_steps - 1 by default, meaning the first step runs at full attention. The first steps set layout and prompt adherence; making step one exact is cheap insurance.
    • sla_method / sla_engine - sla with comfy_kitchen is the path the turbo LoRA was trained against. sol-attn needs no LoRA but wants its own sla_tau. vsa is for models trained with it.

    Every one of these has a tooltip written by someone who measured it, including the ones saying a value didn't help. Read them before you turn dials.

    Installing it

    cd ComfyUI/custom_nodes
    git clone https://github.com/digital-garbage/FunUI
    pip install -r requirements.txt
    

    Or FunUI via ComfyUI-Manager / comfy node install FunUI, then restart and check Settings ▸ About FunUI. The sla_engine: comfy_kitchen path needs comfy-kitchen>=0.2.32 - that's the floor in requirements.txt, and it is compiled against torch, so if pip proposes to move torch as well, stop and read. Missing comfy-kitchen isn't fatal: the Triton kernel runs instead and the status says so.

    Where people get burned

    Quantizing a LoRA and wondering why it's slow. The memory saving holds; the int8 speed-up doesn't apply to layers a LoRA is patched onto. That's all of them, on a "turbo" LoRA.

    Turning SLA on with an 0.5 sparsity "to be safe". You made it slower. The default is a measurement, not a preference.

    CategoryFunUI/Loaders

    Inputs (20)

    NameTypeDefaultDescription
    model_nameCOMBO0 options:
    weight_dtypeCOMBOdefaultHow the weights are stored. 'default' keeps whatever the file already is.
    compute_dtypeCOMBOdefaultWhat the maths runs in, independently of storage.
    attentionCOMBOdefaultAttention backend. Only what this machine can run is listed; 'default' keeps ComfyUI's launch choice.
    fp16_accumulationoptBOOLEANfalseFaster fp16 matmuls where the torch build supports it.
    slaoptBOOLEANfalseBlock-sparse attention for MiniMax H3 -- the inference path lightx2v's SLA turbo LoRA was distilled against, which is why that LoRA gives no speedup on its own. Roughly 3.7x the attention throughput at 768p/15s. Runs ALONGSIDE the attention backend above rather than replacing it: SLA takes H3's long packed self-attention, the chosen backend takes the text refiner, masked calls and any trailing dense steps. Skipped, with the reason, on anything that is not MiniMax H3 or on a machine without Triton.
    sla_sparsityoptFLOAT0.900–0.95Fraction of key blocks skipped while SLA is on (of video cubes for method vsa). 0.90 is validated; 0.85 is lightx2v's own value and ~15% slower. Break-even is around 0.60 -- below that the kernel is SLOWER than dense, so a low value is a loss, not a safe fallback.
    sla_block_sizeoptCOMBO64How many sequence tokens share one key selection. H3 packs audio at 80 rows per second, so a 128-row block forces 1.6s of speech down one attention pattern while the same rows are 3% of a video frame. Use 128 only when the audio does not matter; 32 routes finer still for about the same time. Triton engine only.
    sla_protect_audiooptBOOLEANtrueAlways attend the prompt's words and every audio stream, whatever top-k picks. Audio is ~1% of the packed sequence, so plain top-k regularly drops all of it and the soundtrack degrades while the video still looks fine. Reference images stay sparse: see sla_references.
    sla_min_seq_lenoptINT81920–1000000Sequences shorter than this stay dense. Guards the short text refiner, which must never be sparsified, and low-resolution runs where block selection costs more than it saves.
    sla_dense_last_stepsoptINT00–8Run the last N sampling steps at full attention. 0 matches lightx2v; 1 was tested and did not help, for +20% time.
    int8_convrotoptBOOLEANfalseMiniMax H3 from an ordinary (bf16/fp16) file: store the attention and MLP weights as int8 (rotated first, so no lone outlier spoils the rounding), the way Comfy-Org's own int8_convrot files are. About half the memory and int8 matmuls on GPUs that have them; slightly different pictures. Does nothing to a file that is already quantized or is not H3 (the status says so), and is skipped when a weight dtype is forced above. The memory saving always holds; the speed-up is lost on layers a LoRA is applied to and when compute dtype is forced.
    sla_engineoptCOMBOcomfy_kitchenWhat runs the sparse calls. comfy_kitchen: ComfyUI's compiled int8 sol_attn (needs comfy-kitchen 0.2.32+; the Triton kernel runs instead where it is missing, and the status says so). triton: FunUI's own kernel, the only one with block size and stabilize motion.
    sla_dense_stepsoptSTRING1Steps at full attention, counted from 1: '1' is the first step, '1-2' the first two, '1,6' the first and sixth; blank or '0', none. The first steps set the layout and how closely the prompt is followed, so the first is exact by default. Adds to dense last steps. The log names the steps that ran this way.
    sla_referencesoptCOMBOoffReference images, conditioning frames and the prompt's vision tokens. off: sparse like the video. light: each keeps its best 15% for every query block. heavy: all of them exact, slow with big references. On comfy_kitchen, light picks one 15% for all query blocks.
    sla_tailoptBOOLEANfalsecomfy_kitchen engine: the skipped blocks still count, as one averaged term per query block, instead of not at all. Off matches what the SLA turbo LoRA was trained against; untested on H3.
    sla_stabilize_motionoptBOOLEANfalseTriton engine: each layer leans toward the blocks it picked last step, so near ties stop flipping and showing as a faint double exposure on fast motion. A fix for that one symptom; a little more memory.
    sla_methodoptCOMBOslaHow the blocks worth attending are chosen. sla: a fixed share (sla_sparsity), what SLA turbo LoRAs were trained against. sol-attn: a per-block threshold (sla_tau), no training needed; comfy_kitchen engine. vsa: video cubes, only for models trained for it (the status says when the model lacks VSA's layers); run by ComfyUI's own Model Sparse Attention, keeping 1 - sla_sparsity of the cubes.
    sla_tauoptFLOAT1.300–4sol-attn only: higher skips more. About 1.0 keeps ~16% of key blocks exact, 1.5 ~7%, 2.0 ~3%.
    sla_sparsity_startoptFLOAT0.000–0.95Skip fewer blocks early and more late: the share rises in a straight line from this value on the first step to sla_sparsity on the last (attention concentrates as denoising goes, GSRS, arXiv 2610.21129). 0 is off: one fixed share all run. One H3 render: quality held, perhaps a little better. Try 0.70 with sla_sparsity 0.90. Early steps cost more time than late ones.

    Outputs (2)

    NameTypeDescription
    modelMODEL—
    statusSTRING—