Nucleus-Image Transformer Loader

Every workflow in this pack starts here, so this is the node to get right. Nucleus-Image Transformer Loader is the ComfyUI front door to Nucleus-Image, NucleusAI's sparse mixture-of-experts diffusion transformer. That's a 17B-parameter model with 64 routed experts per layer, but only about 2B params actually compute on any given forward pass. It's a clever trick: the file is huge, the working set is small, and this loader is how you get that distinction to play nice with your VRAM.

The model itself is worth a sentence of context. Nucleus-Image launched in April 2026 as an open, base model - no RLHF, no distillation, no preference tuning - built on flow matching (the pack's own scheduler config is a FlowMatchEulerDiscreteScheduler). That last bit matters more than the hype: it means the old SDXL habits you're carrying over are wrong, and the KB's flow-matching guidance applies here. More on that in the sampler article.

What it actually does

The loader reads a single safetensors file from ComfyUI/models/diffusion_models/ and reconstructs the transformer from a config bundled inside the pack (32 layers, the first three dense, the rest routed MoE with 64 experts). It reads FP8 quantization scales from the file's own metadata and dequantizes the non-expert weights to bf16 as it goes. The experts are the twist: they stay in FP8, and the loader decides per-expert whether they live on the GPU or the CPU offload device. Default of zero blocks swapped means every expert sits on the GPU and you get the fast path; you trade that away with the Block Swap node if VRAM runs short.

Everything non-expert starts on the offload device (your CPU RAM) and only moves to the GPU when the sampler runs. So don't panic when the loader returns instantly and your GPU is idle - that's by design.

The inputs that matter

• model_name - dropdown of everything in models/diffusion_models/. This is where nucleus_image_transformer_fp8.safetensors (≈16.9 GB) goes.
• precision - bf16 (default), fp16, or fp32. The README's tested config uses bf16, and the load path dequantizes toward bf16 under the hood, so leave it there.
• load_device - offload_device (default) or main_device. Only fiddly if you want the non-expert weights pinned to VRAM; the README's 24GB test ran with offload_device.
• block_swap_args - optional BLOCKSWAPARGS input from the Block Swap node. Skip it unless you're fighting OOM.

The single output is NUCLEUS_MODEL, which feeds the Sampler directly, or through Model Shift / CFG Rescale first.

Installing it

This is a pack, so you get all 11 nodes at once. Easiest path is ComfyUI Manager - search "Nucleus-Image" and install the Nucleus-Image-comfyui-beta pack - or:

cd ComfyUI/custom_nodes
git clone https://github.com/a180265/Nucleus-Image-comfyui-beta
# restart ComfyUI

The heavy part is the model. Grab nucleus_image_transformer_fp8.safetensors from the Hugging Face repo and drop it in models/diffusion_models/. There's no requirements.txt in the pack, so you're on the hook for the real dependencies yourself: PyTorch 2.11+ (the loader needs torch.nn.functional.grouped_mm, the MoE kernel), diffusers 0.38+ (nightly if it's not released yet), transformers 4.57+ for Qwen3-VL, plus safetensors, accelerate and scipy. That's a heavier dependency story than most ComfyUI packs - budget for it.

Common issues

• grouped_mm missing - your torch is too old. python -c "import torch; print(hasattr(torch.nn.functional, 'grouped_mm'))" should print True; if not, upgrade.
• OOM at sampling time - you're loading the bf16 file, or your card is smaller than 24GB. Switch to the FP8 file and/or raise blocks_to_swap.
• Don't reach for the built-in UNET/DiffusionModel loader. Nucleus-Image isn't a standard UNet, and the node needs the pack's bundled config plus the FP8 scale handling. The README is explicit: use the dedicated loaders.

One honest caveat: this is a beta, and the author states only FP8 inference was actually run (on an RTX 5090, 24GB). The bf16 file exists but wasn't runtime-tested by them. If your first FP8 generation looks great, that's the verified path - stick with it.