🎚️ Controlnet Apply

ControlNet is the technique that lets a reference image control where things go - edges, depth, a pose - while your prompt still decides what they are. The usual ComfyUI setup for it is a small pile of nodes: load the ControlNet model, run a preprocessor, apply it to your conditioning. SDVN's Controlnet Apply folds that into a single node. You pick the ControlNet model (it can auto-download the popular ones for SD 1.5, SDXL, and Flux), it handles the union type, previews the preprocessed image while it runs, and applies the result to your positive/negative conditioning. One node, the whole flow.

If you've fought with a spaghetti of ControlNet nodes before, the appeal is obvious. The trade is that a lot happens inside one box, so it helps to know what the knobs do.

How it works

ControlNet conditions generation on spatial structure pulled from a reference image, added alongside your prompt rather than replacing it. On modern bases the models are usually union checkpoints - one file that handles every condition, with the mode chosen at inference. That's what the union_type dropdown is: you're telling the union model whether this is a pose, a depth map, a canny edge, a tile, and so on. Strength and the start/end percentages are the standard dials for how hard and for which portion of sampling the control applies.

The inputs and outputs that matter

• image - your control reference.
• control_net - the model (35 options, auto-downloaded if missing; covers SD 1.5, SDXL, Flux, Qwen and more).
• union_type - which condition mode: openpose, depth, canny/lineart/..., tile, repaint, etc., or auto.
• strength (default 1) and start_percent / end_percent - the control dials.
• positive / negative (CONDITIONING, optional) - wire your prompt conditioning through it.

Outputs: modified positive and negative (to your sampler), the preprocessed image (the preview), and parameter - an export for the pack's Auto Generate node so ControlNet settings ride along automatically.

How to install it

Comes with the SDVN pack. ComfyUI Manager, search SDVN_Comfy_node; or:

cd ComfyUI/custom_nodes
git clone https://github.com/StableDiffusionVN/SDVN_Comfy_node
pip install -r custom_nodes/SDVN_Comfy_node/requirements.txt

from the ComfyUI root, then restart. Crucially, the README lists comfyui_controlnet_aux as a required companion - that's what supplies the actual preprocessors (canny, depth, pose detectors). This node's own preprocessor field only offers None / InvertImage; the heavy preprocessing comes from controlnet_aux, and union_type tells the union model which mode to run.

Common issues & troubleshooting

The ControlNet does nothing, or errors on load. ControlNet models are architecture-specific - an SD-era file will not load on Flux, and vice versa. Match the model to your base. This is the single most common ControlNet mistake.

Preprocessors missing. If preprocessing isn't happening, install controlnet_aux (above) and restart. The pack relies on it.

Overcooked results. The old default of strength 1.0 is too strong on modern union models - the community numbers land around 0.65-0.9 (roughly 0.8 for depth, 0.9 for pose on Flux-era unions). A common trick is to end the control partway (end_percent around 0.5) so the later sampling steps stay the model's own. Dial down before you conclude ControlNet "ruined" the image.

Union type mismatch. If you fed a depth map but left union_type on pose, the model applies the wrong interpretation. Set it to match what your control image actually is (or try auto).

Flux inpaint needs the VAE. Some Flux modes (the Alimama inpaint path the README mentions) expect the optional vae input wired in. If an inpaint-style ControlNet misbehaves, connect a VAE.