Diffusers Load ControlNet

If you've used ComfyUI for more than five minutes you already know the native Load ControlNet Model node, so your first question is fair: why does this pack ship its own? The answer is the word "Diffusers" in the name. ComfyUI's built-in loader reads single-file ControlNet checkpoints in Comfy's own format. This node loads them through the HuggingFace diffusers library instead - which means it eats the folder-shaped, config.json-plus-weights format that a lot of research models are actually published in. The ComfyUI_Anytext pack (AnyText, JoyType, Glyph-ByT5) is basically a bridge that runs those diffusers-native pipelines inside Comfy, and this node is the front door for the ControlNet half of that.

The concrete case is JoyType. The pack author describes it as "a typical canny ControlNet implementation by diffusers library," and its weights (jdh-algo/JoyType-v1-1M) ship as a diffusers-format canny ControlNet - a directory, not a .safetensors file. Comfy's native loader won't touch that directory. This one will.

How it works and the two inputs that matter

The node's own description spells out its logic, and it's worth reading literally: load control_net_pretrained if control_net_name is None, else load control_net_name. So you've got two ways in, and they're mutually exclusive:

• control_net_name - a ControlNet you've dropped into ComfyUI/models/controlnet in diffusers format. This is the normal path. Download JoyType-v1-1M into that folder and it shows up here.
• control_net_pretrained - the fallback the node uses when control_net_name is None. Think of it as the "grab it from a pretrained/hub reference" lane for when you haven't got a local copy selected.

Both dropdowns read None on a fresh install because you haven't downloaded anything yet - that's expected, not a bug. The third input, dtype, defaults to auto and lets you force a precision (fp16, bf16, fp32, or one of the fp8 variants). Leave it on auto unless you're chasing a specific VRAM or compatibility problem; fp16 is the safe manual pick. The fp8 options are there for the brave, and not every ControlNet will load cleanly at fp8, so don't reach for them first.

It hands back two outputs: diffusers_control_net, the loaded model object that plugs straight into the pack's Diffusers Apply ControlNet node, and controlnet_name as a plain string, handy for routing or just seeing what you loaded.

Installing it

The node comes with the pack. ComfyUI Manager, search ComfyUI_Anytext, install, restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/zmwv823/ComfyUI_Anytext

Restart ComfyUI. Then the part people skip: the loader is empty until you give it a model. For the JoyType workflow, download the whole jdh-algo/JoyType-v1-1M repo into ComfyUI/models/controlnet. It's a folder of files, so grab the entire thing, not one weight file - a half-downloaded diffusers directory is the number-one reason this node throws on load.

Where people get stuck

The recurring trap is format. If control_net_name stubbornly shows None after you've downloaded something, the odds are you dropped a Comfy-format single-file ControlNet in there and this loader only recognizes diffusers directories - or you unpacked the folder one level too deep. Confirm there's a config.json sitting next to the weights.

Worth a bit of expectation-setting too: JoyType is an SD1.5/SDXL-era canny ControlNet, and canny is exactly the condition every modern base already covers well (see the KB's ControlNet notes). This loader is the right tool if you specifically want JoyType's multilingual text pipeline; it is not a general upgrade over Comfy's native ControlNet stack for everyday canny work. Reach for it when the diffusers format is the whole reason you're here.