NEW 2vXpSwA7: anytest-v4 | openpose-v2_1 || abovzv: segment || bdsqlsz: canny | depth | lineart-anime | mlsdv2 | normal | normal-dsine | openpose | recolor | segment | segmentv2 | sketch | softedge | t2i-color-shuffle | tile-anime-α | tile-anime-β | tile-real || BRIA AI: bg-gen | canny | colorgrid | depth | NEW fill | openpose | recolor || CVL-Heidelberg: canny | depth || destitech: inpaint | inpaintv2 || diffusers: canny small | mid | full | depth small | mid | full | zoe || EcomXL: inpaint | softedge || Eugeoter: NEW noobai canny | depth | lineart-anime | lineart-real | mangaline | normal | scribble-pidi | scribble-hed | softedge-hed | tile | sdxl anime-canny | vidit-depth || h94: ip-adapter | ipa-vith | ipa+ | ipa+face || Hetaneko: pony canny | cannyv2 | color | depth | replicate | replicatev2 || HighCWu: canny-v3 || huchenlei: PuLID || Kataragi: canny | flatline | flatline-lora | inpaint | NEW ipa | line2color | line2color-lora | lineart | lineart-lora | NEW noob-ipa | recolor | recolor-lora | NEW rough-coating | tori29-blur | xdog-sketch || kohya-ss: real blur | canny | depth | anime blur | blur-beta | canny | depth | openpose | openposev2 | replicate | replicatev2 | scribble || PromeAI: lineart || ShermanG lineart || Stability.ai: canny | depth | recolor | revision | sketch || SargeZT: depth-16b-zoe | depth-faid-vidit | depth-zeed | depth-zoe | softedge | t2i-adapters binary | canny | color | depth | segmentation | softedge || TencentARC: canny | depth-midas | depth-zoe | lineart | openpose | recolor | sketch || TheMistoAI: mistoline | mistoline-lora || thibaud: openpose | openpose-lora || ttplanet: tile-real | tile-realv2 || NEW windsingai: pose | tile | tile-10w || xinsir: canny | cannyv2 | depth | openpose | openpose-twins | scribble | scribble-anime | tile | union | union-promax
(13.01.2025 - First NoobAI controlnets uploaded by Eugeoter)
(12.01.2025 - First Illustrious controlnets uploaded: windsingai-pose & -tile)
This collection strives to create a convenient download location of all currently available Controlnet models for SDXL. Please do read the version info for model specific instructions and further resources. If you are the author of one of these models and don't want it to appear here, please contact me to sort this out. The naming scheme of the files follows lllyasviel's from here https://huggingface.co/lllyasviel/sd_control_collection/tree/main.
CAUTION: The variants of controlnet models are marked as checkpoints only to make it possible to upload them all under one version, otherwise the already huge list would be even bigger. If you use downloading helpers the correct target folders are extensions/sd-webui-controlnet/models for automatic1111 and models/controlnet for forge/comfyui.
Requirements for Automatic1111: at least version 1.5.0, better upgrade to the latest version of 1.6.0. + sd-webui-controlnet extension v0.400+. Bugs or weird behaviour might occur. If you encounter any irregularities you can join us on our discord and ask for support or get in contact with the developers of Automatic1111/Controlnet via github/huggingface.
A huge thanks to all the authors, devs and contributors including but not limited to: abovzv, bdsqlsz, BRIA AI, CVL-Heidelberg, destitech, the diffusers institution, Eugeoter, h94, Hetaneko, HighCWu, huchenlei, lllyasviel, kataragi, kohya-ss, Mikubill, PromeAI, SargeZT, ShermanG, Stability.ai, TencentARC, TheMistoAI, thibaud, ttplanet, windsingai and xinsir.
Anyline(?) v4 test model released by 2vXpSwA7 on huggingface
Openpose v2.1 test model released by 2vXpSwA7 on huggingface
Segmentation model released by abovzv on huggingface
https://huggingface.co/abovzv/sdxl_segmentation_controlnet_ade20k
Canny model released by bdsqlsz on huggingface
Depth model released by bdsqlsz on huggingface
Lineart-Anime-Denoise model released by bdsqlsz on huggingface
M-LSD v2 model released by bdsqlsz on huggingface
Normal model released by bdsqlsz on huggingface
Normal dsine model released by bdsqlsz on huggingface
https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite
DW-Openpose model released by bdsqlsz on huggingface
Recolor-Luminance model released by bdsqlsz on huggingface
Segmentation-AnimeFace model released by bdsqlsz on huggingface
Segmentation-AnimeFace v2 model released by bdsqlsz on huggingface
Sketch model released by bdsqlsz on huggingface
Softedge model released by bdsqlsz on huggingface
T2I-Adapter-Color Shuffle model released by bdsqlsz on huggingface
Tile-Anime-α Shuffle model released by bdsqlsz on huggingface
Tile-Anime-β Shuffle model released by bdsqlsz on huggingface
Tile-Realistic model released by bdsqlsz on huggingface
Background Generation model released by BRIA AI on huggingface
https://huggingface.co/briaai/BRIA-2.3-ControlNet-BG-Gen
Canny model released by BRIA AI on huggingface
ColorGrid model released by BRIA AI on huggingface
https://huggingface.co/briaai/BRIA-2.3-ControlNet-ColorGrid
Depth model released by BRIA AI on huggingface
https://huggingface.co/briaai/BRIA-2.3-ControlNet-Depth
Generative Fill model released by BRIA AI on huggingface
https://huggingface.co/briaai/BRIA-2.3-ControlNet-Generative-Fill
Openpose v2.3 model released by BRIA AI on huggingface
Recoloring model released by BRIA AI on huggingface
https://huggingface.co/briaai/BRIA-2.3-ControlNet-Recoloring
Canny model released by CVL-Heidelberg on huggingface
Depth model released by CVL-Heidelberg on huggingface
Inpainting model released by destitech on huggingface
https://huggingface.co/destitech/controlnet-inpaint-dreamer-sdxl
Inpainting v2 model released by destitech on huggingface
https://huggingface.co/destitech/controlnet-inpaint-dreamer-sdxl
Full sized Canny model trained and released by the diffusers organization at huggingface
Medium sized Canny model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0-mid
Small sized Canny model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0-small
Full sized Depth model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0
Medium sized Depth model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0-mid
Small sized Depth model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0-small
Zoe Depth model trained and released by the diffusers organization at huggingface
https://huggingface.co/diffusers/controlnet-zoe-depth-sdxl-1.0
Inpainting model released by EcomXL on huggingface
https://huggingface.co/alimama-creative/EcomXL_controlnet_inpaint
Softedge model released by EcomXL on huggingface
https://huggingface.co/alimama-creative/EcomXL_controlnet_softedge
Canny model for noobai released by Eugeoter on huggingface
Depth model for noobai released by Eugeoter on huggingface
Anime Lineart model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-lineart_anime
Realistic Lineart model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-lineart_realistic
Manga Line model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-manga_line
Normal model for noobai released by Eugeoter on huggingface
Pidinet Scribble model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-scribble_pidinet
HED Scribble model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-scribble_hed
HED Softedge model for noobai released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/noob-sdxl-controlnet-softedge_hed
Tile model for noobai released by Eugeoter on huggingface
Anime Canny model released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/controlnext-sdxl-anime-canny
VIDIT Depth model released by Eugeoter on huggingface
https://huggingface.co/Eugeoter/controlnext-sdxl-vidit-depth
IP-adapter trained and released by h94 at huggingface
IP-adapter trained and released by h94 at huggingface
IP-adapter plus trained and released by h94 at huggingface
IP-Adapter-Plus-Face trained and released by h94 at huggingface
Canny model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-canny-new
Canny model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-canny-newv2
Color model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-color
Depth model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-depth
Replicate model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-replicate
Replicate v2 model released by Hetaneko on huggingface
https://huggingface.co/Hetaneko/pony-replicatev2
Canny v3 model released by HighCWu on huggingface
PuLID is an ip-adapter alike method to restore facial identity. It uses both insightface embedding and CLIP embedding similar to what ip-adapter faceid plus model does. However, there is an extra process of masking out the face from background environment using facexlib before passing image to CLIP. PuLID also uses Eva CLIP instead of normal CLIP. In the attn overrides, PuLID also does something more than IPAdapter, as it zero pads the tensor and adds to ortho of hidden states.
PuLID uses both evaclip embedding and insightface embedding as proj module input. So in order to use the feature, you need to install https://github.com/huchenlei/sd-webui-controlnet-evaclip extension.
ComfyUI implementations:
https://github.com/cubiq/PuLID_ComfyUI provided by cubiq, native ComfyUI implementation
https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by ZHO, diffusers-based implementation
Original repository: https://huggingface.co/huchenlei/ipadapter_pulid
Canny model released by Kataragi on huggingface
https://huggingface.co/kataragi/controlnet_canny
Flatline model released by Kataragi on huggingface
https://huggingface.co/kataragi/flatline
Flatline LoRA model released by Kataragi on huggingface
https://huggingface.co/kataragi/flatline
Inpainting model released by Kataragi on huggingface
https://huggingface.co/kataragi/controlnetXL_inpaint_test
IP-Adapter model released by kataragi on huggingface
Line2Color model released by kataragi on huggingface
https://huggingface.co/kataragi/controlnetXL_line2color
The recommended model is animagineXL3.1, but if generated with a model such as hanamomoponyV1.4, the output can be color rough to anime paint-like. The fp16 version (2.5GB) shows an excellent response in both cases, but the lora version (377MB) does not seem to follow the instructions unless it is the training source model, animagineXL3.1.
Line2Color model released by kataragi on huggingface
https://huggingface.co/kataragi/controlnetXL_line2color
The recommended model is animagineXL3.1, but if generated with a model such as hanamomoponyV1.4, the output can be color rough to anime paint-like. The fp16 version (2.5GB) shows an excellent response in both cases, but the lora version (377MB) does not seem to follow the instructions unless it is the training source model, animagineXL3.1.
Lineart model released by Kataragi on huggingface
https://huggingface.co/kataragi/ControlNet-LineartXL
Lineart LoRA model released by Kataragi on huggingface
https://huggingface.co/kataragi/ControlNet-LineartXL
IP-Adapter model for noobai released by kataragi on huggingface
Recolor model released by Kataragi on huggingface
https://huggingface.co/kataragi/ControlNet-recolorXL
Recolor LoRA model released by Kataragi on huggingface
https://huggingface.co/kataragi/ControlNet-recolorXL
Rough Coating model released by kataragi on huggingface
Tori29 Blur model released by kataragi on huggingface
XDoG Sketch model released by kataragi on huggingface
Control Lllite - Blur model trained and released by kohya-ss at huggingface
Control Lllite - Blur Anime model trained and released by kohya-ss at huggingface
Control Lllite - Blur Anime beta model trained and released by kohya-ss at huggingface
Control Lllite - Canny model trained and released by kohya-ss at huggingface
https://huggingface.co/kohya-ss/controlnet-lllite
Control Lllite - Canny Anime model trained and released by kohya-ss at huggingface
Control Lllite - Depth model trained and released by kohya-ss at huggingface
https://huggingface.co/kohya-ss/controlnet-lllite
Control Lllite - Depth Anime model trained and released by kohya-ss at huggingface
Control Lllite - OpenPose Anime model trained and released by kohya-ss at huggingface
Control Lllite - OpenPose Anime v2 model trained and released by kohya-ss at huggingface
Control Lllite - Replicate anime model trained and released by kohya-ss at huggingface
Control Lllite - Replicate anime v2 model trained and released by kohya-ss at huggingface
https://huggingface.co/kohya-ss/controlnet-lllite
Control Lllite - Scribble Anime model trained and released by kohya-ss at huggingface
Lineart model released by PromeAI on huggingface
https://huggingface.co/promeai/sdxl-controlnet-lineart-promeai
Canny control-lora model trained and released by Stability.ai at huggingface
https://huggingface.co/stabilityai/control-lora#midas-and-clipdrop-depth
Depth control-lora model trained and released by Stability.ai at huggingface
https://huggingface.co/stabilityai/control-lora#midas-and-clipdrop-depth
Recolor control-lora model trained and released by Stability.ai at huggingface
https://huggingface.co/stabilityai/control-lora
Revision control-lora model trained and released by Stability.ai at huggingface
Recolor control-lora model trained and released by Stability.ai at huggingface
https://huggingface.co/stabilityai/control-lora#photograph-and-sketch-colorizer
16bit Zoe Depth model trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/controlnet-sd-xl-1.0-depth-16bit-zoe
Faid Vidit Depth model trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/controlnet-sd-xl-1.0-depth-faid-vidit
Zeed Depth model trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/controlnet-sd-xl-1.0-depth-zeed
Zoe Depth model trained and released by SargeZT at huggingface
Dexined Soft Edge model trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/controlnet-sd-xl-1.0-softedge-dexined
Binary t2i adapter trained and released by SargeZT at huggingface
Canny t2i adapter trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi
Color t2i adapter trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi
Depth t2i adapter trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi
Segmentation t2i adapter trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/t2i-adapter-sdxl-segmentation
Soft Edge t2i adapter trained and released by SargeZT at huggingface
https://huggingface.co/SargeZT/t2i-adapter-sdxl-multi
Lineart model released by ShermanG on huggingface
https://huggingface.co/ShermanG/ControlNet-Standard-Lineart-for-SDXL
Canny t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-canny-sdxl-1.0
MiDaS Depth t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-depth-midas-sdxl-1.0
Zoe Depth t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-depth-zoe-sdxl-1.0
Lineart t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-lineart-sdxl-1.0
Openpose t2i adapter trained and released by TencentARC at huggingface
Recolor t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-recolor-sdxl-1.0
Sketch t2i adapter trained and released by TencentARC at huggingface
https://huggingface.co/TencentARC/t2i-adapter-sketch-sdxl-1.0
MistoLine model released by TheMistoAI on huggingface
https://huggingface.co/TheMistoAI/MistoLine
OpenPose model trained and released by thibaud at huggingface
OpenPose LoRA model trained and released by thibaud at huggingface
Illustrious-XL Openpose test model released by windsingai on huggingface
https://huggingface.co/windsingai/Illustrious-XL-openpose-test
Illustrious-XL Tile model released by windsingai on huggingface
Illustrious-XL Tile (10w) model released by windsingai on huggingface
Canny model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-canny-sdxl-1.0
Canny v2 model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-canny-sdxl-1.0
Depth model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-depth-sdxl-1.0
Openpose model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-openpose-sdxl-1.0
Openpose twins model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-openpose-sdxl-1.0
Scribble model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0
Anime Scribble model released by xinsir on huggingface
https://huggingface.co/xinsir/anime-painter
Tile model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-tile-sdxl-1.0
Union model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-union-sdxl-1.0
Union ProMax model released by xinsir on huggingface
https://huggingface.co/xinsir/controlnet-union-sdxl-1.0