🤗 Diffusers Controlnet Loader

Most ControlNet loaders make you go find the right file, download it, and drop it in the right folder before the dropdown even shows anything useful. This one flips that: pick a condition type from a menu, and the node handles finding the model itself.

What it is and how it works

Straight from the pack's own README: this node automatically detects whether the ControlNet you asked for is already downloaded locally, and if it isn't, pulls it from HuggingFace on the spot. That's a meaningfully different workflow from ComfyUI's own ControlNetLoader, which just lists whatever .safetensors files you've already manually placed in the controlnet models folder and does nothing if the folder's empty.

The 15 condition types on offer - canny, openpose, depth, tile, ip2p, shuffle, inpaint, lineart, mlsd, normalbae, scribble, seg, and three more - line up closely with the original ControlNet 1.1 lineup that shipped for SD 1.5 back in 2023: eleven production models plus a few experimental ones, standardized under Lvmin Zhang's own naming rules. That makes sense given this whole pack is built for SD1.5/SDXL-class checkpoints - you're getting the complete, mature, original ControlNet menu rather than the narrower six-or-seven-condition unions that newer architectures ship with.

The inputs and outputs that matter

• controlnet_model_name (required, enum) - the condition type. This is the whole decision: pick canny for edge-based structure, depth for spatial layout, openpose for figures, tile for detail-preserving upscales, and so on. Each one is a different trained model, not a mode switch on one file.
• controlnet_model_file (optional, enum) - an override slot for pointing at a specific local file variant instead of letting the node auto-resolve one. Leave it alone unless you've got a specific checkpoint you want to force.

Output is a single controlnet (DIFFUSERS_CONTROLNET) - this is not yet a usable conditioning input by itself. It feeds into a Diffusers Controlnet Unit node, which is where you actually attach an image and a strength to it.

Installing it

Part of the ComfyUI-J pack. ComfyUI Manager: search ComfyUI-J, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/Jannchie/ComfyUI-J
cd ComfyUI-J
pip install -r requirements.txt

Where people get burned

The auto-download convenience cuts both ways. The first time you pick a new condition type, expect an unannounced pause while a multi-hundred-megabyte-to-multi-gigabyte model pulls down from HuggingFace - there's no visible progress bar warning you it's about to happen, so a workflow that ran instantly yesterday can suddenly sit there for a minute today just because you switched condition types. It also means this node needs live internet access on first use of each type, which matters if you're running on an isolated or offline box.

Second: this node loads a model, not a conditioning image. It doesn't preprocess anything - there's no canny edge detector or depth estimator bundled in here. You still need to generate your own canny map, depth map, or pose skeleton with a separate preprocessor pack (ComfyUI's ControlNet Auxiliary Preprocessors is the standard one) before wiring it into the Controlnet Unit node downstream. People coming from a workflow that had preprocessing built in sometimes expect this loader to do double duty, and it doesn't.

Third, and worth remembering as you pick from the 15-item menu: a couple of the SD1.5-era exotics people fondly remember from ControlNet's early days - QR-code/brightness conditioning and true "Reference" conditioning - were never rebuilt for anything past SDXL, but they were also never universal even in the SD1.5 lineup itself; check that whichever type you pick is actually one this loader lists before assuming it's covered.