[Inference.Core] Layer Diffuse Apply

Layer Diffuse is the technique behind those "cut-out sticker" generations that come out of the sampler already transparent - no background-removal pass, no rembg, no manual masking. It's a real research result (SIGGRAPH 2024, from the same Lvmin Zhang / lllyasviel who gave the community ControlNet and IC-Light), later ported into ComfyUI by huchenlei as the ComfyUI-layerdiffuse custom node, and this pack bundles its own version of those nodes under the Inference_Core_ prefix. Layer Diffuse Apply is where you turn the whole thing on.

What it actually does

Normal diffusion models predict an RGB latent - three channels, no transparency. Layer Diffuse patches the model so it also learns to encode a fourth, invisible channel: alpha. The patch is small (it hooks into either the model's attention or convolution layers, not a full retrain), which is why this works as a lightweight "apply" node rather than requiring a whole different checkpoint. Once patched, you sample completely normally - same KSampler, same prompt - and the resulting latent secretly carries transparency information that a later decode step (Inference_Core_LayeredDiffusionDecode) pulls back out as a real alpha mask.

The inputs that matter

• config - picks the architecture and injection method: SDXL, Attention Injection, SDXL, Conv Injection, or SD15, Attention Injection, attn_sharing. This has to match the actual base your checkpoint is built on. Between the two SDXL flavors, which one holds up better is checkpoint-dependent - if attention injection gives you weird color bleeding or a halo around the transparent edge, try conv injection instead, and vice versa.
• weight (-1 to 3, default 1) - how strongly the transparency patch is applied. Lower it if the patch is degrading normal image quality more than you're willing to tolerate; raise it if edges aren't separating cleanly from the background.
• model - your regular checkpoint, unpatched, going in.

The single output is a patched MODEL - from here your graph looks completely ordinary right up until decode. Nothing else in the sampling chain changes.

Installing it

Comes bundled with the pack. ComfyUI Manager: search "ComfyUI-Inference-Core-Nodes". Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/LykosAI/ComfyUI-Inference-Core-Nodes

then install.py, or pip install -e .[cuda] (swap in [cuda12], [rocm], [directml], or [cpu] for your hardware) and restart. One thing worth knowing before your first run: like the upstream layerdiffuse project it's built from, expect it to reach out and download its own conversion weights from Hugging Face the first time you actually apply the patch. Give it a working connection and a minute or two the first time - after that it's cached.

Where people get stuck

The config-vs-checkpoint mismatch is the one that actually bites: pick an SDXL config against an SD1.5 checkpoint (or the reverse) and you don't get a subtly wrong image, you get an immediate tensor-shape crash. Double-check your config selection matches your loaded model before troubleshooting anything else.

This is genuinely a niche technique - real and still used, but the volume of people talking about it is small and steady rather than trending, and it's mostly people who've found a specific use for hard, clean cutouts (game assets, sticker packs, product renders that need a transparent PNG immediately) and keep coming back to it rather than casual users trying it once. If that's your use case, it's worth the setup; if you just need to remove a background from an image you already have, a dedicated background-removal node (BiRefNet, rembg) is simpler and doesn't require matching your ControlNet to your checkpoint's base architecture.

Apply on its own only gets you the patched model - you still need Inference_Core_LayeredDiffusionDecode downstream to actually turn the sampled latent into a usable image-plus-alpha pair.