🤗 Diffusers Generator

This is the node the whole pack exists to build up to. The README's pitch is that a stock ComfyUI equivalent of this setup takes seven nodes - checkpoint loader, two CLIP text encodes, empty latent, KSampler, VAE decode, and whatever ControlNet plumbing you've bolted on. This one node does all of it: sampling, ControlNet application, inpainting, reference-only style transfer, and the VAE decode, and it hands you back a finished image.

What it is and how it works

Under the hood it's still a standard diffusion sampling loop - noise prediction guided by your prompt embeddings, refined over a number of steps, following whatever CFG scale you set. What's different is how much extra conditioning got folded directly into this one node instead of being spread across a graph: feed it a source image and it does img2img; add a mask and it inpaints only the masked region; attach ControlNet units and it conditions on spatial structure; flip on reference-only and it borrows a style from a reference image using self-attention injection instead of any trained conditioning model at all.

That last one is worth a beat of context. Reference-only isn't a true ControlNet - there's no conditioning image being fed through a preprocessor, no trained control model. It works by injecting the reference image's own self-attention features into the generation process, which was a genuinely novel zero-shot consistency trick when it first landed in the SD1.5 era, before IP-Adapter and later instruction-editing models displaced it as the go-to approach on newer architectures. Since this whole pack targets SD1.5 and SDXL checkpoints exclusively, reference-only here isn't a legacy holdover bolted onto something newer - it's exactly the technique's native habitat.

The inputs and outputs that matter

The required list is long because this node does the job of several stock nodes, but only a handful actually need your attention on a first pass:

• pipeline, positive_prompt_embedding, negative_prompt_embedding - the pipeline from a Diffusers Pipeline node, and the two embeddings from a Diffusers Compel Prompt Embedding node. Non-negotiable, every run needs all three.
• num_inference_steps (default 30), guidance_scale (default 7, i.e. CFG) - the usual sampling knobs.
• strength (default 1, range 0–1) - only matters once you feed it a source image via images. At 1 it's effectively txt2img (ignores the source almost entirely); lower it toward 0 to keep more of the original image and change less.
• seed, batch_size (up to 16), width/height (up to 8192, though your actual VRAM ceiling will land well before that) - standard generation controls.
• reference_strength / reference_style_fidelity (defaults 1 and 0.5) - only do anything when reference_only is enabled below.

Optional inputs are where the real power lives:

• images (optional) - a source image, turning this into img2img. Combine with strength below 1.
• mask (optional) - paired with images, restricts changes to the masked region for inpainting.
• controlnet_units (optional) - a single Controlnet Unit or a Controlnet Unit Stack, for spatial conditioning.
• reference_image, reference_only, reference_only_adain (all optional) - the reference-only style transfer trio. reference_only (enable/disable) turns the feature on; reference_only_adain (enable/disable) adds AdaIN normalization on top for a stronger style match - the README describes these as the two supported modes, plain attention injection versus attention plus AdaIN.

Output is images (IMAGE), directly usable - no separate decode step required, which is the whole point.

Installing it

Part of the ComfyUI-J pack. ComfyUI Manager: search ComfyUI-J, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/Jannchie/ComfyUI-J
cd ComfyUI-J
pip install -r requirements.txt

Where people get burned

There's no LoRA slot anywhere on this node, consistent with the rest of the pack - it's still an open TODO item in the README. If your usual generation setup leans on a character or style LoRA, that piece has no home here.

The schema's width/height cap of 8192 and batch_size cap of 16 are declared limits, not a promise your GPU can actually deliver them - treat them as the node's ceiling, not a recommendation. And combining everything at once (source image plus mask plus ControlNet units plus reference-only) is exactly the pack's own "Change Clothes" demo workflow from the README - it's a real, supported combination, but it's also the setup most likely to produce a confusing result if one piece is misconfigured, since you're stacking four different conditioning mechanisms on a single sampling pass. If output looks wrong with everything enabled, disable pieces one at a time rather than guessing which one's the culprit.