Diffusion Edge (batch size ↑ => speed ↑, VRAM ↑)

Most edge detectors are fast, cheap, and a little crude. DiffusionEdge is the opposite: it runs an actual diffusion model to produce edge maps, and the output is noticeably cleaner and more coherent than what Canny or HED give you - crisp, continuous lines with less noise. The catch is right there in the node's own name, "batch size ↑ => speed ↑, VRAM ↑." This is the heavyweight option. When you want the best possible edge map and you've got the VRAM budget to spend, this is the one. When you just need a quick canny pass, it's overkill.

How it works

Instead of running a hand-tuned filter over pixels, DiffusionEdge treats edge detection as a generative task - a diffusion model trained specifically to output edge maps. That's the same "use a diffusion prior for a perception task" trick that makes Marigold produce such clean depth maps: the model has a strong sense of what real, coherent edges look like, so it fills in continuous lines where a classical detector would sputter and break. The tradeoff is that you're running a real model, with real VRAM and time cost, instead of a millisecond filter. You feed the resulting edge map into an edge/lineart-style ControlNet to condition your generation on that structure.

The inputs that matter

The node takes an image and returns an IMAGE - the edge map - for your ControlNet Apply node. A few options shape it:

• environment (default indoor; also urban, natrual) - picks which trained checkpoint runs. The model ships three variants tuned for different scene types, so match it to your subject: interiors, cityscapes, or natural/outdoor scenes. (Yes, "natrual" is spelled that way in the node - that's the actual option string, not a typo to fix.)
• patch_batch_size (default 4, range 1–16) - the speed/VRAM dial the node name is warning you about. The image gets processed in patches; a bigger batch does more of them at once, so it's faster but eats more VRAM. If you're hitting out-of-memory errors, drop this. If you've got headroom and want it done quicker, raise it.

resolution (default 512) sets the working size.

Installing it

Through ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. On first run it downloads the Diffusion-Edge checkpoint matching your environment choice (indoor, urban, or natural) from HuggingFace - these are real model files, not the tiny annotator weights, so the first pull takes a moment.

Where people get burned

The number one issue is VRAM. This preprocessor is genuinely hungry, and if it OOMs, your first move is to drop patch_batch_size down toward 1 - that trades speed for a smaller memory footprint and usually gets it running on a modest card. Second, pick the right environment checkpoint; running the indoor model on a forest scene works but you'll get a better edge map from the matching one. And be honest with yourself about whether you need this at all - for a lot of workflows a plain Canny or HED map conditions the generation just as well at a fraction of the cost. Reach for DiffusionEdge when edge quality is the thing you're actually chasing, not as a default. As always, the edge map does nothing without a matching ControlNet model loaded downstream.