JoyAI_Image_SM_Clip

Everything about JoyAI-Image flows through this loader, because the model's "text encoder" is a full vision-language model - Qwen3-VL - and you load it here, the same way you'd load a checkpoint's CLIP. Wire its clip output into JoyAI_Image_ENCODER for conditioning and into JoyAI_Image_Understand for the VLM chat mode. It's the smallest node in the pack and often the one that decides whether the whole thing runs at all.

The interface is brutally simple: two dropdowns, clip and gguf, and you pick a file in one of them and set the other to none. The clip combo lists files in ComfyUI/models/clips - that's where JoyAI-Image-Und-merger_bf16.safetensors goes, the merged bf16 text encoder from the smthem HF repo. The gguf combo lists ComfyUI/models/gguf, which is where JoyAI-Image-Und-merger-Q6_K.gguf lives. Both load the same weights; the difference is memory and how much you care about the encoder's precision.

Why the GGUF option is the smart default

Here's the thing the pack README is gesturing at with its "use less memory" note: the text encoder is a large fraction of this model's total footprint. On modern LLM-encoded models, the encoder is often the VRAM bottleneck rather than the diffusion transformer itself - quantize the encoder hard and keep the precision in the DiT is the standing advice, and this node is the dial for exactly that.

The Q6_K GGUF sits in the sweet spot: comfortably below the bf16 encoder's multi-GB weight, with a visible but modest quality cost that mostly shows up in text rendering and fine detail adherence. Q6 is a defensible line for most people. The loader does the GGUF dance through diffusers' quantizer, which means you need the gguf Python package (≥0.10.0) in your environment - pip install -r requirements.txt handles it if the requirements resolved, but it's worth knowing the dependency exists.

How it behaves at load time

Every loader node in this pack calls clear_comfyui_cache() first, which unpins any ComfyUI models and empties the GPU cache. Good hygiene on paper, slightly rude in practice - every time you re-run the loader it evicts whatever else was loaded. Don't put it on a timer node or in a loop and expect smooth sailing.

Note that both combos default to "none", so the node fires fine with neither picked - and then gives you nothing. If your sampler suddenly has no conditioning, check that you actually selected a file here rather than leaving the default.

Install and files

Shared pack install, once:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_JoyAI_Image
cd ComfyUI_JoyAI_Image
pip install -r requirements.txt

Then restart ComfyUI. Grab the encoder from the merged repo (https://huggingface.co/smthem/JoyAI-Image-Edit-merge-dit-gguf) - either JoyAI-Image-Und-merger_bf16.safetensors → models/clips, or JoyAI-Image-Und-merger-Q6_K.gguf → models/gguf. The requirements pin transformers>=4.57.0,<4.58.0, and since this node loads the encoder through transformers, that pin matters: if another custom node forces a different transformers version, this loader is often the first thing that breaks. The community flagged exactly that conflict back when JoyAI-Image ComfyUI support landed - budget for it if you run a big custom-node zoo.