Safe Latent Diffusion Guider (Schramowski et al. 2023)

Here's what a normal negative prompt does: it replaces the empty prompt in CFG's unconditional pass, so the sampler is pushed away from your negative everywhere, at every step, at full strength. That's the whole mechanism, and it's why negatives overshoot - you ask for "no plants" on a garden prompt and the model starts sanding down greenery that the prompt asked for.

Safe Latent Diffusion, from Schramowski et al. (CVPR 2023), does something smarter. It only steers away from the concept at the pixels where the image is already moving toward it, and it ramps in after a warm-up. It's the difference between a blanket ban and a targeted fix. The paper built it to suppress NSFW content on unfiltered models; in daily use it's a very good "remove this specific thing" knob.

The mechanism

d_c is the positive prediction, d_u the empty-prompt one, and d_n the prediction for the concept you want gone (your negative prompt).

• A mask: mu = clamp(s_S * |d_c - d_n|, max=1) where the difference (d_c - d_n) is below a threshold lam, otherwise 0. In words: only where the concept is already influencing the image.
• The steering direction gamma = mu * (d_n - d_u), plus a momentum term that accumulates the correction across steps so it isn't a fresh fight every step.
• For the first few steps gamma is ignored entirely (the warm-up), because early steps are where layout is decided and heavy steering there wrecks composition.
• Result: d_u + w (d_c - d_u - gamma).

That warm-up is the part you'll notice. In the pack's own comparison, the plants only start disappearing after about step 10 of 50, and the layout from the no-negative run survives.

Inputs and output

It's a guider, not a model patch, so it goes into SamplerCustomAdvanced, not KSampler.

• model, positive, negative, null - and yes, you wire three conditioning inputs. negative is the concept to steer away from; null is the true unconditional, i.e. an empty CLIPTextEncode ("The empty prompt: the true unconditional"). Getting these two confused is the number-one way to make this node look broken.
• cfg - 7 by default. This node's own guidance scale, independent of any KSampler.
• preset - medium, strong, max: the paper's own configurations. Medium is warm-up 10, the strongest masking threshold, and moderate momentum; strong starts at step 7 with a heavier momentum; max has no warm-up at all and the hardest mask.
• space - auto (the method's own), which for this paper means the noise prediction.

Output is a GUIDER, wired into SamplerCustomAdvanced's guider input. Any other CFG Megapack stages you've installed on the model still apply - the guider only replaces the combine stage.

Install

Manager → search CFG Megapack → install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/AbstractEyes/comfy-cfg-megapack

No dependencies beyond ComfyUI itself, no model files. Needs a current ComfyUI (comfy_api.latest; the README says tested on 0.38.0 with torch 2.11) - on older builds the whole pack fails to import.

Where people get burned

Cost. With SLD you're running three model passes per step instead of two, since the concept prediction is its own forward pass. Expect noticeably slower sampling. It's the price of a targeted negative, and it's why most people don't leave it on for every render.

It does nothing at CFG 1. Same trap as every negative prompt: at CFG 1 the guidance term collapses and ComfyUI skips the unconditional pass, so there is nothing to subtract from. Guidance-distilled models that want cfg 1 want positive phrasing instead, or an attention-level negative method.

preset max is not a gentle upgrade. Warm-up 0 means it starts steering from the first step, at the hardest masking, so layout and colour shift. If you like the composition of the run without a negative, stay on medium.

It only works in SamplerCustomAdvanced. KSampler has no guider input; the node will sit in your graph doing nothing. The pack's CFG Guider: Positive, Negative and Null and the other five guider nodes in the same folder follow the same rule.

It won't fix a bad negative. SLD changes when and where your negative acts, not what it means. If "plants, potted plant, leaves" gives you a worse image than no negative at all on a garden prompt, that's a prompting problem, not a guider problem - try naming a benign replacement ("plain painted backdrop") or drop the negative entirely.

Sanity check the wiring fast: put a classic CFG guider and this one on the same seed and prompt. They should differ only after the warm-up, and the SLD one should look like the no-negative run with the concept quietly missing.