Apply Qwen Image DiffSynth ControlNet

Qwen-Image made its name as the editing family - Qwen-Image-Edit became the default open instruction-editing stack because you hand it a picture and a sentence and the change happens, no mask required. But instruction editing has a weakness: it follows words, not lines. If you need a pose, a depth structure, or a specific composition to survive the edit, you want a controlnet. This node applies the Qwen-Image DiffSynth ControlNet, which brings canny, depth, and inpainting control to the Qwen-Image generation model.

The controlnet came to ComfyUI native support in August 2025, announced alongside Qwen LoRA support and the first context-window nodes. It loads as a MODEL_PATCH via ModelPatchLoader (the DiffSynth controlnet files), and this node fuses it into your model with a control image to steer generation. It's the same code path that the Z-Image Fun ControlNet uses - same subclass, Qwen-Image-specific handling.

How it works

The DiffSynth controlnet weights are a patch, not a standalone network, so the apply node's job is to wire the control image into the model's forward pass. It encodes your control image into the latent space with the VAE you provide, then patches the model's double blocks so each denoising step is steered toward the structure of the control map, scaled by strength.

The optional mask input is where the inpainting mode lives: give it a mask and the controlnet focuses its conditioning on the masked region, which is how you get "regenerate only this part, keeping everything else." The node flips the mask internally (1.0 - mask) for the controlnet's convention.

Inputs and what to set

• model - your Qwen-Image model.
• model_patch - the MODEL_PATCH from ModelPatchLoader (the DiffSynth controlnet file, from the DiffSynth-Studio releases).
• vae - encodes the control image; must match the model.
• image - the control image (canny map, depth map, etc.). Required.
• strength - float, default 1.0, range -10 to 10. Full control at 1.0; 0.3-0.7 for loose guidance.
• mask - optional, switches on inpainting conditioning.

Output: one MODEL for the sampler. The controlnet file itself goes in ComfyUI/models/model_patches/.

Common issues & troubleshooting

Mismatched tensor errors are the signature failure. The early Qwen controlnet threads are full of "size of tensor a (X) must match tensor b (Y)" errors. Nine times out of ten it's a control image at the wrong resolution or the wrong VAE. Your control image needs to match the latent resolution the sampler expects - resize the control map to the generation resolution.

Control is strong at 1.0 - consider a step cutoff. Like the Z-Image controlnet from the same DiffSynth lineage, holding full strength across every step can fight the base model's quality on distilled paths. If output looks "controlled but stiff," drop the strength or release the condition in the last few steps.

Don't expect it on Qwen-Image-Edit. This controlnet targets the generation model, not the edit model. If you're doing instruction edits and want structure control, you chain the controlnet before the edit stage rather than asking the edit model to honor it.