Load ControlNet Model (diff) (Any)

This node exists because of how ControlNet was originally distributed, back when Lvmin Zhang first published it in February 2023. Some early releases weren't complete, standalone weight files - they were a diff: a delta relative to the base SD checkpoint's weights. To reconstruct the actual usable ControlNet, you needed the base model on hand to add that delta back onto. DiffControlNetLoader_Any is the loader built for exactly that case, which is why it's the only ControlNet loader in this pack that takes a MODEL as a required input alongside the file itself.

Why it needs a model input

Compare it to the regular ControlNetLoader_Any: that one just loads a self-contained ControlNet file and hands back CONTROL_NET. This node needs two things - model (the base diffusion model the diff was trained against) and control_net_name (the diff file itself) - because the file on disk isn't the full ControlNet. It's the difference between the base model's weights and the ControlNet-augmented weights, and this loader adds that difference back on top of whatever you feed into model to reconstruct the real thing. Feed it the wrong base model and you're not loading a broken ControlNet - you're loading a ControlNet reconstructed against the wrong architecture entirely, which won't error cleanly so much as just not work right.

Where this format sits today

The ControlNet ecosystem has moved decisively away from diff-format releases. Since 2024, distribution standardized on complete "union" checkpoints - one full file per architecture covering every condition, with no need to reconstruct anything against a base model. That shift happened because supply itself sped up dramatically: a union checkpoint now lands within about a week of a new base release, versus the year SDXL originally waited for coverage. None of that newer union tooling is diff-format. Practically, this node is for a specific, aging slice of files: ControlNet .pth releases from the original 2023 SD 1.5 wave that were published as diffs rather than complete weights. If what you downloaded is a modern union or full-weight file, you want ControlNetLoader_Any instead - this node will either reject it or handle it incorrectly, since it expects a delta, not a complete set of weights.

The any input

Same optional input every loader in this pack carries: any, which accepts anything and is never used for anything except delaying execution until whatever's wired into it finishes. Useful here too - this node depends on your base model already being loaded (it needs it to reconstruct the ControlNet), so there's a natural, real dependency to stage even beyond the pack's usual VRAM-timing trick.

Installing it

ComfyUI Manager: search ComfyUI-LoaderUtils, install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/lrzjason/ComfyUI-LoaderUtils

No extra dependencies.

Where people get burned

The core mistake is using this node at all when a regular full-weight ControlNet would do - check where the file came from before assuming it's diff-format; most things released in the last two years aren't. The second mistake is feeding the wrong base model into model: the diff has to be added onto the exact architecture it was trained as a delta from, so mismatching that produces a ControlNet that loads without complaint but conditions incorrectly, which is a much harder failure to diagnose than an outright error.