Nodes/ComfyUI-Advanced-ControlNet/Load Advanced ControlNet Model (diff) πŸ›‚πŸ…πŸ…’πŸ…
ComfyUI Node Runs on cloud

Load Advanced ControlNet Model (diff) πŸ›‚πŸ…πŸ…’πŸ…

The model-aware ControlNet loader

By KosinkadinkΒ·Created 3 years agoΒ·Updated 8 days agoΒ· 993
Load Advanced ControlNet Model (diff) πŸ›‚πŸ…πŸ…’πŸ…
  • model
  • _tk_opt
  • CONTROL_NET
β—„cnetβ–Ύβ–Ί

This is the "diff" flavor of the Advanced ControlNet loader - the one that takes your model as an input. Most of the time you don't want it. The regular Load Advanced ControlNet Model loads a standalone ControlNet file and you're done. The diff version exists for a specific breed of ControlNet that was trained as a difference against the base model and needs the model's own weights to reconstruct itself at load time.

The pack author is refreshingly blunt about this in the README: some ControlNets are designed to receive the model, and if you don't know what that does, you probably don't want to use the diff version. That's the honest guidance. If your ControlNet loads fine in the normal loader, use the normal loader.

When the diff loader is the right call

A handful of older SD 1.5 "difference" ControlNets ship as the delta between a control-trained UNet and the base checkpoint rather than as a complete model. To rebuild them you have to add that delta back onto the base weights - which is why this node needs a MODEL input the plain loader doesn't. Feed it the wrong kind of ControlNet (a normal full one) and you'll get errors or garbage guidance, so this really is a load-what-it-asks-for situation.

Whatever it loads comes out as a fully Advanced ControlNet, so all the pack's scheduling and weighting features are available downstream regardless of which loader you used. The loader is just the on-ramp; the features live on the Apply node and the keyframe/weight nodes.

Inputs and outputs

Required:

  • model - the base model the diff ControlNet is computed against. It has to match the architecture the ControlNet was trained for.
  • cnet - a dropdown of the ControlNet files in ComfyUI/models/controlnet. Pick the diff-style one.

Optional:

  • _tk_opt - a Timestep Keyframe you can bake onto the ControlNet here. Per the README, if you're using an Apply Advanced ControlNet node downstream you almost never need this; set your timestep keyframes on the Apply node instead. It's here for the edge case where this loader isn't feeding an Advanced Apply.

Output is CONTROL_NET - the loaded Advanced ControlNet, wired into the control_net input of your Apply node.

Installing the pack

ComfyUI Manager: Install Custom Nodes β†’ "ComfyUI-Advanced-ControlNet" β†’ restart. Or cd ComfyUI/custom_nodes && git clone https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet and restart. It's a clean, pure-ComfyUI pack from Kosinkadink (the AnimateDiff-Evolved author), so no dependency drama. The pack doesn't include preprocessors (that's comfyui_controlnet_aux) or any ControlNet model files - those go in ComfyUI/models/controlnet yourself.

Common gotchas

The main one is using this node when you don't need it. If your ControlNet isn't a diff/difference model, the plain Load Advanced ControlNet Model is correct and this one may fail to load it. Reach for the diff loader only when a model's docs say it needs the base weights.

Second, the model you feed in must match the ControlNet's architecture - an SD 1.5 diff ControlNet wants an SD 1.5 model. Mismatched architectures are a common source of shape errors at load. And if the dropdown is empty, your files aren't in ComfyUI/models/controlnet (or ComfyUI hasn't been restarted since you added them).

CategoryAdv-ControlNet πŸ›‚πŸ…πŸ…’πŸ…

Inputs (3)

NameTypeDefaultDescription
modelMODELβ€”
cnetCOMBO0 options:
_tk_optoptTIMESTEP_KEYFRAMEβ€”

Outputs (1)

NameTypeDescription
CONTROL_NETCONTROL_NETβ€”