ControlNetXL (CNXL)

NEW 2vXpSwA7: anytest-v4 | openpose-v2_1 || abovzv: segment || bdsqlsz: canny | depth | lineart-anime | mlsdv2 | normal | normal-dsine | openpose | recolor | segment | segmentv2 | sketch | softedge | t2i-color-shuffle | tile-anime-α | tile-anime-β | tile-real || BRIA AI: bg-gen | canny | colorgrid | depth | NEW fill | openpose | recolor || CVL-Heidelberg: canny | depth || destitech: inpaint | inpaintv2 || diffusers: canny small | mid | full | depth small | mid | full | zoe || EcomXL: inpaint | softedge || Eugeoter: NEW noobai canny | depth | lineart-anime | lineart-real | mangaline | normal | scribble-pidi | scribble-hed | softedge-hed | tile | sdxl anime-canny | vidit-depth || h94: ip-adapter | ipa-vith | ipa+ | ipa+face || Hetaneko: pony canny | cannyv2 | color | depth | replicate | replicatev2 || HighCWu: canny-v3 || huchenlei: PuLID || Kataragi: canny | flatline | flatline-lora | inpaint | NEW ipa | line2color | line2color-lora | lineart | lineart-lora | NEW noob-ipa | recolor | recolor-lora | NEW rough-coating | tori29-blur | xdog-sketch || kohya-ss: real blur | canny | depth | anime blur | blur-beta | canny | depth | openpose | openposev2 | replicate | replicatev2 | scribble || PromeAI: lineart || ShermanG lineart || Stability.ai: canny | depth | recolor | revision | sketch || SargeZT: depth-16b-zoe | depth-faid-vidit | depth-zeed | depth-zoe | softedge | t2i-adapters binary | canny | color | depth | segmentation | softedge || TencentARC: canny | depth-midas | depth-zoe | lineart | openpose | recolor | sketch || TheMistoAI: mistoline | mistoline-lora || thibaud: openpose | openpose-lora || ttplanet: tile-real | tile-realv2 || NEW windsingai: pose | tile | tile-10w || xinsir: canny | cannyv2 | depth | openpose | openpose-twins | scribble | scribble-anime | tile | union | union-promax

ControlNetXL (CNXL) - A collection of Controlnet models for SDXL

(13.01.2025 - First NoobAI controlnets uploaded by Eugeoter)
(12.01.2025 - First Illustrious controlnets uploaded: windsingai-pose & -tile)

This collection strives to create a convenient download location of all currently available Controlnet models for SDXL. Please do read the version info for model specific instructions and further resources. If you are the author of one of these models and don't want it to appear here, please contact me to sort this out. The naming scheme of the files follows lllyasviel's from here https://huggingface.co/lllyasviel/sd_control_collection/tree/main.

CAUTION: The variants of controlnet models are marked as checkpoints only to make it possible to upload them all under one version, otherwise the already huge list would be even bigger. If you use downloading helpers the correct target folders are extensions/sd-webui-controlnet/models for automatic1111 and models/controlnet for forge/comfyui.

Requirements for Automatic1111: at least version 1.5.0, better upgrade to the latest version of 1.6.0. + sd-webui-controlnet extension v0.400+. Bugs or weird behaviour might occur. If you encounter any irregularities you can join us on our discord and ask for support or get in contact with the developers of Automatic1111/Controlnet via github/huggingface.

A huge thanks to all the authors, devs and contributors including but not limited to: abovzv, bdsqlsz, BRIA AI, CVL-Heidelberg, destitech, the diffusers institution, Eugeoter, h94, Hetaneko, HighCWu, huchenlei, lllyasviel, kataragi, kohya-ss, Mikubill, PromeAI, SargeZT, ShermanG, Stability.ai, TencentARC, TheMistoAI, thibaud, ttplanet, windsingai and xinsir.

ControlNetXL (CNXL) diff-canny-small

Small sized Canny model trained and released by the diffusers organization at huggingface

https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0-small

ControlNetXL (CNXL) diff-depth-full

Full sized Depth model trained and released by the diffusers organization at huggingface

https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0

ControlNetXL (CNXL) diff-depth-small

Small sized Depth model trained and released by the diffusers organization at huggingface

https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0-small

ControlNetXL (CNXL) huchenlei-PuLID

PuLID is an ip-adapter alike method to restore facial identity. It uses both insightface embedding and CLIP embedding similar to what ip-adapter faceid plus model does. However, there is an extra process of masking out the face from background environment using facexlib before passing image to CLIP. PuLID also uses Eva CLIP instead of normal CLIP. In the attn overrides, PuLID also does something more than IPAdapter, as it zero pads the tensor and adds to ortho of hidden states.

PuLID uses both evaclip embedding and insightface embedding as proj module input. So in order to use the feature, you need to install https://github.com/huchenlei/sd-webui-controlnet-evaclip extension.

ComfyUI implementations:

Original repository: https://huggingface.co/huchenlei/ipadapter_pulid

ControlNetXL (CNXL) kataragi-line2color

Line2Color model released by kataragi on huggingface
https://huggingface.co/kataragi/controlnetXL_line2color

The recommended model is animagineXL3.1, but if generated with a model such as hanamomoponyV1.4, the output can be color rough to anime paint-like. The fp16 version (2.5GB) shows an excellent response in both cases, but the lora version (377MB) does not seem to follow the instructions unless it is the training source model, animagineXL3.1.

ControlNetXL (CNXL) kataragi-line2color-lora

Line2Color model released by kataragi on huggingface
https://huggingface.co/kataragi/controlnetXL_line2color

The recommended model is animagineXL3.1, but if generated with a model such as hanamomoponyV1.4, the output can be color rough to anime paint-like. The fp16 version (2.5GB) shows an excellent response in both cases, but the lora version (377MB) does not seem to follow the instructions unless it is the training source model, animagineXL3.1.

ControlNetXL (CNXL) kohya-canny

Control Lllite - Canny model trained and released by kohya-ss at huggingface

https://huggingface.co/kohya-ss/controlnet-lllite

ControlNetXL (CNXL) kohya-depth

Control Lllite - Depth model trained and released by kohya-ss at huggingface

https://huggingface.co/kohya-ss/controlnet-lllite

ControlNetXL (CNXL) kohya-replicate-anime-v2

Control Lllite - Replicate anime v2 model trained and released by kohya-ss at huggingface

https://huggingface.co/kohya-ss/controlnet-lllite

ControlNetXL (CNXL) SAI-recolor

Recolor control-lora model trained and released by Stability.ai at huggingface

https://huggingface.co/stabilityai/control-lora

ControlNetXL (CNXL) SargeZT-t2i-canny

Canny t2i adapter trained and released by SargeZT at huggingface

https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi

ControlNetXL (CNXL) SargeZT-t2i-color

Color t2i adapter trained and released by SargeZT at huggingface

https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi

ControlNetXL (CNXL) SargeZT-t2i-depth

Depth t2i adapter trained and released by SargeZT at huggingface

https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi

ControlNetXL (CNXL) SargeZT-t2i-softedge

Soft Edge t2i adapter trained and released by SargeZT at huggingface

https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi