UniVidX • Load Model

UniVidX's loader is the 24 GB VRAM gatekeeper

By dreamrec·Created 4 months ago·Updated 4 months ago· 78

model

◄variantintrinsic►

◄dtypebfloat16►

◄compile_ditfalse►

◄prefer_sage_attnfalse►

◄vram_buffer_gb4.0►

◄step_distill_loranone►

◄step_distill_strength1.00►

◄dit_weight_modefp8_prequantized►

This is the node that decides whether you can run UniVidX at all. UniVidXLoader loads a 14-billion-parameter video diffusion model - Wan2.1-T2V-14B with UniVidX's per-modality LoRA adapters bolted on - and hands it to the sampler over a purple UNIVIDX_MODEL socket. If you Googled this node, you almost certainly just found out it needs 24 GB of VRAM minimum. That's the honest part; let's deal with it.

UniVidX is a SIGGRAPH 2026 model that decomposes a video clip into intrinsic channels (RGB / albedo / irradiance / normal) or alpha channels (matte / foreground / background). This pack wraps its official pipeline as a ComfyUI graph, and this loader is the front door. It's the first of the five nodes in the flagship workflow: Loader → Task Mode → Sampler → Decode → Save. Nothing else happens until this node finishes loading.

The hardware reality, up front

Don't install this on a 12 GB card and hope. The pack's own README is refreshingly blunt about it: 16 GB OOMs, 12 GB flat-out can't run, and pre-Ampere (RTX 20-series) is out entirely because Wan's Flash-Attention-2 path needs compute capability 8.0+. The path that fits on 24 GB is the FP8 one, which this loader now defaults to (dit_weight_mode=fp8_prequantized): the DiT drops from ~28 GB to ~14 GB. On a 32 GB card you get the BF16 option too. And you'll need ~85 GB of disk, because the base model alone is a 69 GB Hugging Face download.

How it works

The loader does the standard BF16 cold-load of Wan's six DiT shards, then - when you pick the FP8 mode - walks the DiT's ~400 linear layers, computes per-tensor absmax scales, and replaces each one with an FP8Linear that dequantizes on forward. UniVidX's LoRA adapters stay in BF16 by walking through the PEFT wrappers. Models are cached per (variant, ckpt, device, dtype, vram_buffer, fp8_qtype, compile_dit, prefer_sage_attn, dit_weight_mode), so touching any of those knobs triggers a full re-load. That cold-load is the long wait - budget minutes, not seconds, and it re-does it whenever you change a setting.

The inputs that matter

variant - intrinsic or alpha. This is the big fork: it must match the family of the mode you pick in Task Mode, or the sampler refuses to run.
dit_weight_mode - keep the 0.5.0 default fp8_prequantized; bf16_shards is the older 28 GB path, and fp8_runtime_experimental is an escape hatch known to hang. The legacy dtype=fp8_* option was removed for the same reason.
dtype - bfloat16 matches UniVidX's training and is the right default; float16 is fine on most Ada/Blackwell cards.
vram_buffer_gb - GB kept free for activations. 4.0 is a sane default; lower it only to squeeze residency, raise it only when you hit OOM. It's effectively a no-op on the FP8 path.
step_distill_lora - lightx2v merges a step-distillation LoRA that unlocks a ~3-5x faster preview mode (4 steps, cfg 1.0). It's experimental on decomposition targets - see below.
compile_dit / prefer_sage_attn - the pack's own 0.5.0 benchmarks show both making things slower on the FP8 path. Leave them off unless you enjoy debugging.

Install

cd ComfyUI/custom_nodes
git clone --recurse-submodules https://github.com/dreamrec/UniVidX_ComfyUI.git
cd UniVidX_ComfyUI
python -m pip install -r requirements.txt
python install.py        # creates the model-path junction/symlink

The --recurse-submodules matters - it pulls the ~500 MB UniVidX vendor repo, which is part of the mechanism, not optional bloat. Then the models:

pip install -U "huggingface_hub[cli]"
hf download Wan-AI/Wan2.1-T2V-14B  --local-dir ComfyUI/models/wan21_t2v_14b
hf download houyuanchen/UniVidX    --local-dir ComfyUI/models/unividx

ComfyUI Manager can install the pack too (search "UniVidX"), but it won't fetch the models for you. You need ComfyUI 0.20+, Python 3.10+, and - on Blackwell - torch 2.7 with cu128. macOS is out (no CUDA).

Troubleshooting

MissingModelFile - you skipped a hf download. Re-run both.
CUDA error: no kernel image is available - your torch is too old for your GPU. Upgrade to torch>=2.7+cu128.
Cold-load hangs forever - you're on the legacy FP8 runtime-quantize path. Switch dit_weight_mode to fp8_prequantized.

One honest caveat: this is a small, fresh pack, and I found zero community threads about UniVidX in the Reddit corpus this site tracks - every benchmark in the README is the author's own. The PSNR numbers are internally consistent and the pack is unusually well-documented, but treat the "9.43 minutes on an RTX 5090" claims as a starting estimate, not gospel. If you have the VRAM, it's worth the download; if you don't, rent an L40 on RunPod for a few dollars an hour and skip the hardware grief.

CategoryUniVidX

Inputs (8)

Name	Type	Default	Description
variant	COMBO	intrinsic	2 options: intrinsic, alpha
dtype	COMBO	bfloat16	Compute dtype. bfloat16 (default) matches UniVidX's training. float16 is functionally equivalent on most Blackwell/Ada cards. fp8_e4m3fn / fp8_e5m2 post-quantize the DiT via optimum-quanto's qfloat8 path: halves DiT memory (~28 GB → ~14 GB), enables Marlin FP8 matmul on Hopper/Blackwell. EXPERIMENTAL — the quantize() pass over Wan2.1-14B + UniVidX's LoRA stack is slow (10+ min) and may hang on some configurations; if your run stalls on cold-load, fall back to bfloat16.
compile_ditopt	BOOLEAN	false	Run torch.compile(dit, mode='reduce-overhead', dynamic=True) after model load. First sampler step is 60-120 sec slower (graph capture), subsequent steps are typically 20-30% faster. Best for long runs at fixed resolution; loses its compile cache when you change resolution/frame-count. Cached separately per (variant, dtype, compile) tuple so toggling triggers a re-load + re-compile.
prefer_sage_attnopt	BOOLEAN	false	If True (and `sageattention` is installed in the ComfyUI venv), monkey-patch DiffSynth's Wan DiT attention chain so SageAttention wins over Flash Attention 2. SageAttention's INT8-quantized attention is typically faster on Hopper/Blackwell (claimed 2-5x by upstream, our measurements vary). On Blackwell (sm_120) FA3 isn't available — FA2 is the default winner — so this flag is the main attention-backend lever. No-op if sageattention isn't installed.
vram_buffer_gbopt	FLOAT	4.00–96	GB of VRAM kept free for activations / KV cache / VAE decode. Passed to UniVidX's pipeline-level enable_vram_management(), which wraps text encoder + DiT + VAE through DiffSynth's offload helper so layers live on CPU and stream to GPU on demand. Higher = more headroom but more streaming (slower). Lower = more residency. 4.0 GB is a sane default for BF16 Wan2.1-14B + UniVidX LoRAs on 32 GB cards. Cached per (variant, dtype, vram_buffer, ...) so two loader nodes with different values get distinct model instances.
step_distill_loraopt	COMBO	none	Merge a step-distillation LoRA into the DiT base weights at load time, enabling near-production-quality decompositions at 4-6 sample steps + cfg_scale=1. Cuts wall-time per chunk by ~3-5x. Currently supports 'lightx2v' (Wan2.1-T2V-14B-StepDistill-CfgDistill-Lightx2v, rank-64). EXPERIMENTAL: step-distill quality on UniVidX's per-modality decompositions (Albedo / Irradiance / Normal / Alpha) is unverified - LightX2V was trained on natural-image content, not on synthetic decomposition targets. RGB-style outputs are most likely to retain quality; Normal and Alpha mattes are highest risk. Pairs with `step_distill_strength`. Requires the file under models/loras/lightx2v/ (see FileNotFoundError message for download cmd).
step_distill_strengthopt	FLOAT	1.000–2	Effective merge strength for the step-distill LoRA. 1.0 = standard merge (recommended for the distillation effect); 0.0 = no merge (equivalent to step_distill_lora='none'); >1.0 = overdrive (may produce artifacts but worth testing if 1.0 gives weak step-reduction effect on your content). Cached as part of the model key, so changing this triggers a full reload.
dit_weight_modeopt	COMBO	fp8_prequantized	How the DiT weights are stored after load. 0.5.0 default: `fp8_prequantized`. `fp8_prequantized` (recommended): the DiT's ~400 Linears + biases + norms are converted to FP8 e4m3fn after the standard BF16 cold-load. Two implementation paths share this label — (a) FILE-BASED if a Kijai `Wan2_1-T2V-14B_fp8_e4m3fn_scaled.safetensors` is present under models/diffusion_models/ (no such file exists upstream for Wan2.1 as of 0.5.0; reserved forward-compat slot), (b) RUNTIME-QUANTIZE otherwise (the actual 0.5.0 path on every Wan2.1 install today). Quality: PSNR ≥ 30 dB per modality vs BF16 reference. Wall: 9.43 min on R2AIN_video, 13% faster than bf16_shards, ~14 GB DiT VRAM. `bf16_shards`: standard BF16 path — ~28 GB DiT in VRAM (with vram_buffer streaming layers as needed). The 0.3.x baseline. Pixel-for-pixel identical to UniVidX's vanilla output but slower. `auto` (legacy): preserved for back-compat with old saved workflows. In 0.5.0+ this resolves to `fp8_prequantized` (was `bf16_shards` in 0.4.0). `fp8_runtime_experimental`: legacy mmgp.offload.quantize() pass after BF16 cold load — known to hang on this stack. Kept only as an escape hatch for users who need to replicate pre-0.4.0 quirks; will be removed in 0.6.0.

Outputs (1)

Name	Type	Description
model	UNIVIDX_MODEL	—