Nodes/ComfyUI-DiffSynth-Studio/ControlNetPathLoader
ComfyUI Node

ControlNetPathLoader

Seven SD 1.5 ControlNets behind one dropdown

By AIFSH·Created 2 years ago·Updated 2 years ago· 101
ControlNetPathLoader
    • ControlNetConfigUnit
    model_id
    scale0.50
    control_net_name

    ControlNetPathLoader is a small node that does one useful thing: it turns a dropdown choice into a ready-to-use ControlNet unit for the pack's toon-shading pipeline. You won't use it with ComfyUI's normal ControlNet nodes - it speaks this pack's own ControlNetConfigUnit type, which DiffutoonNode accepts on its three optional controlnet1/controlnet2/controlnet3 inputs.

    The menu

    The model_id dropdown offers seven conditions: canny, depth, softedge, lineart, lineart_anime, openpose, and tile. That's the classic SD 1.5 ControlNet-v1-1 set, and it's worth knowing what each one actually does before you pick:

    • canny - hard, thin edges. Best when you want the structure locked down tightly.
    • lineart / lineart_anime - clean line drawings. The anime variant is the one you want if your source is anime-style art or you want that clean-ink look.
    • softedge - gentler gradient edges, more forgiving than canny on noisy footage.
    • openpose - human skeleton/pose keypoints. Keeps a person's motion and stance while the style gets replaced.
    • depth - spatial layout. Good when the geometry matters more than the edges.
    • tile - feeds the source image back at low resolution as the condition. It's the "keep the composition and colors, re-render the details" option, and it's the backbone of the shipped Diffutoon workflow.

    How it works

    Pick a model_id and it maps to the matching control_v11*.pth file from lllyasviel's ControlNet-v1-1 repo, downloads it into your ComfyUI/models/controlnet folder if it isn't there, and packages it as a ControlNetConfigUnit with your scale attached. That's the node's output - ControlNetConfigUnit - which is what the Diffutoon pipeline knows how to load alongside the checkpoint and motion module.

    The scale input defaults to 0.5, which is on the loose end of the classic guidance range. In the bundled Diffutoon workflow, lineart and tile run at 0.5 each - the reasoning being that toon shading wants the structure of your source video preserved while the SD 1.5 checkpoint plus AnimateDiff do the style work. Crank scale toward 1.0 if the result drifts too far from your footage; drop it if the video gets stiff and tracey.

    There's an optional control_net_name that lists your local controlnet folder, but it only short-circuits the download when the local file matches the exact filename that node would have fetched. Pick any other file and it downloads its own copy anyway.

    Gotchas

    • SD 1.5 only. These are v1-1 files for the SD 1.5 UNet, and they're consumed by DiffSynth's own pipeline. Don't try to reuse them with SDXL or Flux ControlNets - different architecture, different files entirely.
    • The download is the quiet part. First use of any model_id triggers a synchronous download from HuggingFace into your controlnet folder. The UI freezing for a bit is normal.
    • If you're in mainland China and HuggingFace is unreachable, note the asymmetry: SDPathLoader has a ModelScope toggle, but ControlNetPathLoader does not - it's HuggingFace-only, and its README-famous first-run failure mode is exactly that.

    Install comes free with the pack - ComfyUI Manager (search "ComfyUI-DiffSynth-Studio") or git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio into custom_nodes, pip install -r requirements.txt, restart. ffmpeg on your PATH is required for the video side. It's a fiddly little node, but it's also the reason the Diffutoon workflow can hold your video's structure instead of melting it into generic anime mush.

    CategoryAIFSH_DiffSynth-Studio

    Inputs (3)

    NameTypeDefaultDescription
    model_idCOMBO7 options: canny, depth, softedge, lineart, lineart_anime, openpose, +1
    scaleFLOAT0.50
    control_net_nameoptCOMBO0 options:

    Outputs (1)

    NameTypeDescription
    ControlNetConfigUnitControlNetConfigUnit