Nodes/ComfyUI_PromptStudio/Prompt Studio Apply Qwen Structure
ComfyUI Node

Prompt Studio Apply Qwen Structure

The ControlNet Step Your Qwen Graph Is Missing

By tiko13·Created 3 months ago·Updated 12 days ago· 8
Prompt Studio Apply Qwen Structure
  • model
  • vae
  • guide
  • inpaint_image
  • inpaint_mask
  • MODEL
◄strength0.00►
◄control_modelQwen-Image-2.1-Fun-Controlnet-Union.safetensors►

Qwen 2.1 doesn't take a ControlNet checkpoint the way SD1.5 did. It takes a model patch - a several-gigabyte union file that has to be loaded and applied to your diffusion model before the sampler ever sees it. This node is that application step, and it's the piece people forget to wire when they wonder why their pose guide does nothing.

What it is

PS_ApplyQwenStructure sits between your model chain and the sampler. You give it the Qwen model, a compatible vae, a prepared guide image and a strength, and it hands back a MODEL with structural conditioning baked in. Structurally it's the Qwen-specific twin of "ControlNetApply" - the pack is explicit that other model families need their own application node, and that the reusable, model-agnostic parts (Reference Collection, Reference Item, Structure Input) keep working regardless.

Why you'd want it: post-Flux ControlNet supply got solved and the menu got narrower. As the ControlNet guide puts it, a union checkpoint now lands within about a week of each new base and covers canny, depth, pose, an edge model, scribble and gray plus an inpainting mode - the SD-era exotics never got rebuilt. Qwen 2.1's Fun ControlNet Union is exactly that shape: one file, four conditions, and it also carries an inpaint mode. Guidance is also the one thing instruction editing never fully replaced when several people or objects must land in specified places.

How it works

Read the source and it's refreshingly blunt:

  • If strength is 0 or guide is None, it returns your model untouched. No patch load, no VRAM spent, no error. An empty Structure Input upstream is therefore a legal, free no-op.
  • Otherwise it searches models/model_patches for the filename in control_model, and requires exactly one match - either the exact registered name including subfolder, or an unambiguous bare filename. Zero or two matches and it refuses to guess: Select an installed, unambiguous Qwen structure model.
  • It loads the patch with ComfyUI's own ModelPatchLoader and applies it with ZImageFunControlnet, passing your vae, the guide image and strength. (Yes, the class is named for Z-Image; it's the same Alibaba-PAI Fun control stack and it's what ComfyUI ships to consume these union patches.)
  • The optional inpaint_image and inpaint_mask are a matched pair. Connect one without the other and you get Connect both inpaint image and mask for background preservation - that pair is how native pose + inpainting works, and it comes straight off PS_StructureInput.

The inputs that matter

  • strength - connect PS_StructureInput's strength output rather than typing it. 0 disables ControlNet entirely; up toward 1 pins the structure harder. The Studio authors that number, so wiring it keeps graph and UI in agreement.
  • control_model - the 7.5 GB Qwen-Image-2.1-Fun-Controlnet-Union.safetensors in model_patches by default. Change it only if you have a different registered Qwen union patch; a non-Qwen patch will not load. The output MODEL goes to the sampler's model input.

Install

The node comes with the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/tiko13/ComfyUI_PromptStudio

Restart ComfyUI, then let Settings → Setup wizard → Run setup fetch the union patch into model_patches (about 7.5 GB, verified against a pinned checksum - budget disk). Setup also offers the optional extractors for pose, depth and sketch; those come from Fannovel16's ComfyUI_ControlNet_Auxiliary_Preprocessors pack, which the wizard installs through ComfyUI Manager and which needs its own restart. Native edge detection needs no extractor at all.

Qwen weights - the ControlNet included - sit under the Qwen Research License, so the wizard asks you to accept a strict non-commercial research/evaluation notice before downloading. It's not a click-through formality you can ignore if you're generating for money.

Common issues

  • Nothing happens, no error. Almost always strength = 0 or an empty guide. That's the designed pass-through - check the guide card in Studio actually has an image and a non-zero strength. If you're running the graph from the canvas rather than through Studio, the reference_json widget on Structure Input is what populates the guide; an empty one means no guide.
  • "Select an installed, unambiguous Qwen structure model." The control_model string didn't match exactly one file. If you have two copies in model_patches subfolders, the node correctly refuses; delete one or rename.
  • Patch loads but the guide is ignored. Check the guide is a map compatible with the patch family, and that you're feeding the union patch something it was trained on. A hand-made JPEG edge map will technically pass through and produce something, just not the something you wanted.
  • Porting this to another model family. Don't. Apply Qwen Structure only applies Qwen patches; use your family's own applier and keep feeding it the guide and strength from Structure Input.
CategoryPrompt Studio/References

Inputs (7)

NameTypeDefaultDescription
modelMODELQwen diffusion MODEL to condition. Insert this node before the sampler; connect the output to the sampler's model input.
vaeVAEVAE compatible with the Qwen model, normally the same one used for encoding references and decoding the result.
guideIMAGEConnect Structure Input's guide output or a compatible prepared guide image. None passes the model through without loading a control patch.
strengthFLOAT0.000–1Usually connect Structure Input's strength output. 0 disables ControlNet; increasing toward 1 strengthens structural conditioning. Zero or a missing guide returns the original model without loading the patch.
control_modelSTRINGQwen-Image-2.1-Fun-Controlnet-Union.safetensorsInstalled compatible Qwen union ControlNet patch in model_patches. Use its exact registered name including subfolder, or an unambiguous filename. Settings > Setup can install the default. Loaded only when a guide is active.
inpaint_imageoptIMAGE—
inpaint_maskoptMASK—

Outputs (1)

NameTypeDescription
MODELMODELQwen MODEL with active guide conditioning, or the original MODEL if the guide is disabled. Connect to the sampler.