Prompt Studio Apply Qwen Structure
The ControlNet Step Your Qwen Graph Is Missing
- model
- vae
- guide
- inpaint_image
- inpaint_mask
- MODEL
Qwen 2.1 doesn't take a ControlNet checkpoint the way SD1.5 did. It takes a model patch - a several-gigabyte union file that has to be loaded and applied to your diffusion model before the sampler ever sees it. This node is that application step, and it's the piece people forget to wire when they wonder why their pose guide does nothing.
What it is
PS_ApplyQwenStructure sits between your model chain and the sampler. You give it the Qwen model, a compatible vae, a prepared guide image and a strength, and it hands back a MODEL with structural conditioning baked in. Structurally it's the Qwen-specific twin of "ControlNetApply" - the pack is explicit that other model families need their own application node, and that the reusable, model-agnostic parts (Reference Collection, Reference Item, Structure Input) keep working regardless.
Why you'd want it: post-Flux ControlNet supply got solved and the menu got narrower. As the ControlNet guide puts it, a union checkpoint now lands within about a week of each new base and covers canny, depth, pose, an edge model, scribble and gray plus an inpainting mode - the SD-era exotics never got rebuilt. Qwen 2.1's Fun ControlNet Union is exactly that shape: one file, four conditions, and it also carries an inpaint mode. Guidance is also the one thing instruction editing never fully replaced when several people or objects must land in specified places.
How it works
Read the source and it's refreshingly blunt:
- If
strengthis 0 orguideis None, it returns your model untouched. No patch load, no VRAM spent, no error. An empty Structure Input upstream is therefore a legal, free no-op. - Otherwise it searches
models/model_patchesfor the filename incontrol_model, and requires exactly one match - either the exact registered name including subfolder, or an unambiguous bare filename. Zero or two matches and it refuses to guess:Select an installed, unambiguous Qwen structure model. - It loads the patch with ComfyUI's own
ModelPatchLoaderand applies it withZImageFunControlnet, passing yourvae, theguideimage andstrength. (Yes, the class is named for Z-Image; it's the same Alibaba-PAI Fun control stack and it's what ComfyUI ships to consume these union patches.) - The optional
inpaint_imageandinpaint_maskare a matched pair. Connect one without the other and you getConnect both inpaint image and mask for background preservation- that pair is how native pose + inpainting works, and it comes straight off PS_StructureInput.
The inputs that matter
strength- connect PS_StructureInput'sstrengthoutput rather than typing it. 0 disables ControlNet entirely; up toward 1 pins the structure harder. The Studio authors that number, so wiring it keeps graph and UI in agreement.control_model- the 7.5 GBQwen-Image-2.1-Fun-Controlnet-Union.safetensorsinmodel_patchesby default. Change it only if you have a different registered Qwen union patch; a non-Qwen patch will not load. The outputMODELgoes to the sampler's model input.
Install
The node comes with the pack:
cd ComfyUI/custom_nodes
git clone https://github.com/tiko13/ComfyUI_PromptStudio
Restart ComfyUI, then let Settings → Setup wizard → Run setup fetch the union patch into model_patches (about 7.5 GB, verified against a pinned checksum - budget disk). Setup also offers the optional extractors for pose, depth and sketch; those come from Fannovel16's ComfyUI_ControlNet_Auxiliary_Preprocessors pack, which the wizard installs through ComfyUI Manager and which needs its own restart. Native edge detection needs no extractor at all.
Qwen weights - the ControlNet included - sit under the Qwen Research License, so the wizard asks you to accept a strict non-commercial research/evaluation notice before downloading. It's not a click-through formality you can ignore if you're generating for money.
Common issues
- Nothing happens, no error. Almost always
strength= 0 or an empty guide. That's the designed pass-through - check the guide card in Studio actually has an image and a non-zero strength. If you're running the graph from the canvas rather than through Studio, thereference_jsonwidget on Structure Input is what populates the guide; an empty one means no guide. - "Select an installed, unambiguous Qwen structure model." The
control_modelstring didn't match exactly one file. If you have two copies inmodel_patchessubfolders, the node correctly refuses; delete one or rename. - Patch loads but the guide is ignored. Check the guide is a map compatible with the patch family, and that you're feeding the union patch something it was trained on. A hand-made JPEG edge map will technically pass through and produce something, just not the something you wanted.
- Porting this to another model family. Don't. Apply Qwen Structure only applies Qwen patches; use your family's own applier and keep feeding it the guide and strength from Structure Input.
Inputs (7)
| Name | Type | Default | Description |
|---|---|---|---|
| model | MODEL | Qwen diffusion MODEL to condition. Insert this node before the sampler; connect the output to the sampler's model input. | |
| vae | VAE | VAE compatible with the Qwen model, normally the same one used for encoding references and decoding the result. | |
| guide | IMAGE | Connect Structure Input's guide output or a compatible prepared guide image. None passes the model through without loading a control patch. | |
| strength | FLOAT | 0.000–1 | Usually connect Structure Input's strength output. 0 disables ControlNet; increasing toward 1 strengthens structural conditioning. Zero or a missing guide returns the original model without loading the patch. |
| control_model | STRING | Qwen-Image-2.1-Fun-Controlnet-Union.safetensors | Installed compatible Qwen union ControlNet patch in model_patches. Use its exact registered name including subfolder, or an unambiguous filename. Settings > Setup can install the default. Loaded only when a guide is active. |
| inpaint_imageopt | IMAGE | — | |
| inpaint_maskopt | MASK | — |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| MODEL | MODEL | Qwen MODEL with active guide conditioning, or the original MODEL if the guide is disabled. Connect to the sampler. |