JoyAI_Image_SM_KSampler

This is the workhorse - the pack's own KSampler, because JoyAI-Image won't run through the stock one. It takes the MODEL from JoyAI_Image_SM_Model, the latents from JoyAI_Image_LATENTS, the conditioning from JoyAI_Image_ENCODER, and grinds through the flow-matching schedule until a denoised latent comes out the other side. Everything fun and everything expensive happens here.

The architecture is a 40 double-block MMDiT - a heavily reworked Wan 2.1-family transformer - sampling with a FlowMatchDiscreteScheduler (shift 4.0) and a guidance_scale that defaults to 5.0. That number is the classifier-free-guidance dial: it pushes the sample away from the empty-prompt "unconditional" direction that the ENCODER's negative output represents. If your edits come out half-hearted, nudge guidance_scale up a bit; if things look overcooked or glitchy, bring it down. Flow-matching models are much more sensitive to this knob than SDXL ever was.

Inputs worth knowing

• model, latents - from the pack's own loaders. Don't mix in a stock checkpoint; the sampler expects the JoyAI-Image DiT and its 16-channel latent format.
• steps - default 20. This is a distilled-ish flow model, not SD1.5; 20 is a reasonable floor, and you'll see diminishing returns past ~30.
• guidance_scale - default 5.0, range 1–20. The single most impactful quality dial here.
• offload (default true) - true keeps the DiT on CPU and streams blocks to the GPU through a block manager. This is the memory strategy that makes the model runnable on 12–16GB cards at all; leave it on unless you have headroom to spare.
• offload_block_num (default 0, effective 1) - how many transformer blocks the GPU manager moves at a time. It's a coarse VRAM/RAM tradeoff knob. 0 silently becomes 1. Tune it only if you're chasing OOM errors, and mostly leave it alone.
• positive / negative (optional) - from the ENCODER. Here's the trap: if you leave them unconnected, the sampler reads cached embeddings from your output directory instead. Handy for caching, but it means a KSampler with no conditioning does not silently no-op - it throws "No backup prompt embeddings found. Please run JOY_SM_ENCODER node first." if the cache is missing.

The memory reality

Read the README's note honestly: the GGUF route still needs on the order of 40GB of system RAM, and the bf16 everything route wants a serious GPU. ComfyUI's own memory management doesn't cover this pack - the offload is the pack's own block manager, and community reports from when JoyAI-Image support first landed confirm you can't rely on the usual low-VRAM rescues. If you're on 16GB VRAM, that's workable with the GGUF DiT and offload on; 8GB is going to hurt.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_JoyAI_Image
cd ComfyUI_JoyAI_Image
pip install -r requirements.txt

restart, and make sure the DiT (safetensors in models/diffusion_models or joy_image_transformer-Q8_0.gguf in models/gguf) is in place. flash-attn>=2.8.0 in the requirements is not decorative - the attention backend is hardcoded to flash-attn, so a failed flash-attn build means this node won't run no matter how many other packs are happy.

Where people get burned

First-run symptom checklist: OOM on the sampler → turn offload on and prefer the Q8_0 GGUF DiT; conditioning error → check the ENCODER ran (or its cache exists); washed-out or weak edits → raise guidance_scale toward 7-8 before touching steps. And if you changed the image but kept the seed and the same latent size, the sampler still runs - you just get a boringly similar result, which is the seed doing its job.