JoyAI_Image_SM_Model

Every workflow starts here. JoyAI_Image_SM_Model loads the actual diffusion transformer - the thing that does the image generation - and hands you a MODEL for the pack's KSampler. It's the pack's stand-in for ComfyUI's stock checkpoint loader, and like the rest of this pack it gives you two ways in: a plain safetensors file or a quantized GGUF.

The model itself is the reason this node exists. JoyAI-Image's transformer is a 40 double-block MMDiT with a 4096 hidden size and a 16-channel latent space - architecturally it's a heavily retrained Wan 2.1-class network (it keeps Wan's VAE and latent format) but with a Qwen3-VL text encoder grafted on, and it's big. A bf16 safetensors of a 40-block 4k-width DiT will embarrass a 12GB card on its own, which is why the README's whole update story is about GGUF.

The two combos, one file

• dit - lists files in ComfyUI/models/diffusion_models. Put joy_image_transformer.safetensors there for the full-precision path.
• gguf - lists files in ComfyUI/models/gguf. Put joy_image_transformer-Q8_0.gguf there for the quantized path.

Pick a file in one dropdown and set the other to none. The node then picks whichever path is real and builds the model through the pack's runtime, defaulting to CPU placement so the sampler can block-offload.

Which should you use? Q8 is basically fp16 at roughly half the size - the quality hit is close to invisible and you save a solid chunk of RAM. The README's "less memory" bullet is about exactly this: with the GGUF DiT plus the quantized text encoder, the whole stack runs in the neighborhood of 40GB of system RAM instead of comfortably more. If the bf16 file fits your rig, it's marginally better; for everyone else, Q8_0 is the honest choice. This mirrors how the wider community has handled every big DiT since Flux - GGUF is the default low-VRAM path, not the compromise.

Gotchas

The loader calls clear_comfyui_cache() before loading, which unpins whatever ComfyUI had in VRAM. It also takes a while - a 40-block DiT, even quantized, is a lot of weights to read off disk and place. That's normal; don't assume it froze. And if you switch between dit and gguf mid-session, expect a fresh long load and a RAM spike while the old model's memory drains.

Also note the file naming: the merged GGUF repo keeps the transformer as joy_image_transformer-Q8_0.gguf and ships JoyAI-Image-Und-merger-Q6_K.gguf - the latter is the text encoder, not the DiT. Drop the right file in the right folder, or your loader dropdown will show both and the wrong pick will silently fail downstream.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_JoyAI_Image
cd ComfyUI_JoyAI_Image
pip install -r requirements.txt

restart ComfyUI, then grab joy_image_transformer-Q8_0.gguf (recommended) or joy_image_transformer.safetensors from https://huggingface.co/smthem/JoyAI-Image-Edit-merge-dit-gguf. The GGUF path needs the gguf Python package (diffusers loads it through its own quantizer), and the requirements pin transformers>=4.57,<4.58 plus flash-attn>=2.8.0 - the flash-attn build is the one that bites people on Windows.

One last thing: this is a young model with a community-built ComfyUI story, and the pack is a port, not the official integration. If the loader errors out on a fresh install, the usual suspects are a transformers version that drifted off the pin or a half-downloaded weight file - re-check those before you suspect the pack.