Eric Qwen-Image Composite Harmonize
The ControlNet pass that makes a pasted collage read as one shot
- pipeline
- controlnet
- composite_image
- control_image
- upscale_vae
- image
- control_image_used
Eric Qwen-Image Harmonize is the ControlNet-flavored answer to the same problem the Edit Unify node solves with Qwen-Edit: a composited image - subject pasted on a background, lighting mismatched, seams visible - that needs to look like one photograph. But where Unify runs the whole frame through the 20B edit model, Harmonize takes the generation model (Qwen-Image base) plus the InstantX ControlNet Union and does a controlled img2img harmonization pass. Different tools, same goal: make the paste invisible.
The wiring tells you its shape immediately: it needs a QWEN_IMAGE_PIPELINE (from the Qwen-Image Loader - the tooltip is explicit that it must be the base, NOT Qwen-Image-Edit), a QWEN_IMAGE_CONTROLNET (from the ControlNet Loader, Union model), and the composite_image. Then the pipeline's own QWEN_IMAGE_PIPELINE is fed by ImageComposer output or any pasted image.
How it works
The clever part is that you don't have to extract a control map yourself. cn_mode defaults to auto_canny - it runs cv2 Canny edge detection on the composite and uses those edges to lock structure while the model re-renders lighting and texture. auto_soft_edge (Sobel+Laplacian) is the smoother alternative; external lets you wire your own map (depth, pose, custom canny) into control_image. So the ControlNet's job here is to hold the composition still while denoise controls how much re-rendering happens:
denoise0.20-0.30 - very light, just color/grain matching.denoise0.30-0.45 - typical harmonization: lighting + soft contact shadows (default 0.42).denoise0.45-0.60 - stronger relight, may shift small details.denoise0.70+ - nearly full re-render; only with strong CN structure.
The refine pass
The default has a two-pass design: refine_pass (True) runs a second, lighter diffusion pass on the result of pass 1 to sharpen detail - skin pores, eyelashes, the stuff that separates "harmonized" from "AI-smoothed." The nice mechanism detail is that the pass-1 latent stays on the GPU (no VAE re-encode round-trip), so detail isn't lost between passes. The refine controls (refine_denoise ~0.18-0.28, refine_steps, refine_cfg, refine_cn_strength) are all pre-tuned; if the output looks soft, raise refine_cfg a touch.
Inputs worth touching
prompt- if empty, a generic harmonization preamble is used. The tooltip's strong recommendation: wire it from the Qwen ControlNet Prompt Rewriter with the composite as input, which lists every subject in place so Qwen doesn't delete them. That's the difference between "unified scene" and "background with the subject quietly removed."negative_prompt- defaults to the official Qwen-Image negative (the Chinese one).cn_strength- 1.0 standard; over 1.5 over-constrains.cn_end- 0.7 default, stopping CN guidance before the final steps so texture isn't over-constrained.steps(40 default; effective denoise steps = steps × denoise),true_cfg_scale(2.5-3.5 best for harmonization),max_mp(working resolution; output is upsampled back to composite size),max_sequence_length(1024, fine to leave).upscale_vae- optional, the 2× Wan upscale VAE from Eric Qwen Upscale VAE Loader, for a sharper 2× final decode with no extra diffusion.upscale_vae_keep_2xkeeps the 2× output when on.
Outputs are image and control_image_used (handy for debugging - you can preview exactly what edges the model saw).
When to reach for it
Composites where the subject's structure is already right and only lighting/texture need unifying. Compared to Unify, Harmonize is the more surgical, denoise-controlled option (and doesn't re-emit the whole frame as aggressively). It's also genuinely good at contact shadows - that ground_contact-style realism the compositing crowd obsesses over. Install: ComfyUI Manager search "Eric Qwen Edit", or git clone https://github.com/EricRollei/Eric_Qwen_Edit_Experiments into custom_nodes/; needs the Qwen-Image base model and the ~2.3 GB Union ControlNet on first load.
Inputs (30)
| Name | Type | Default | Description |
|---|---|---|---|
| pipeline | QWEN_IMAGE_PIPELINE | From the Qwen-Image loader (Qwen-Image base, NOT Qwen-Image-Edit). | |
| controlnet | QWEN_IMAGE_CONTROLNET | From the Qwen-Image ControlNet Loader. Use the InstantX Union model. | |
| composite_image | IMAGE | The composited image to harmonise (e.g. ImageComposer output). | |
| prompt | STRING | Description of the unified scene. If empty, a generic harmonisation preamble is used. RECOMMENDED: wire from the Qwen ControlNet Prompt Rewriter with the composite as input — it will list every subject in place so Qwen cannot delete them. | |
| cn_modeopt | COMBO | auto_canny | How to obtain the control image: auto_canny: cv2 Canny edges of the composite (default, robust). auto_soft_edge: cv2 Sobel+Laplacian soft edges (smoother textures). external: use the wired control_image input (depth, pose, custom canny, etc.). |
| control_imageopt | IMAGE | External control image. Used only when cn_mode = external. | |
| canny_lowopt | INT | 1000–255 | Low threshold for cv2.Canny (auto_canny only). |
| canny_highopt | INT | 2000–255 | High threshold for cv2.Canny (auto_canny only). |
| denoiseopt | FLOAT | 0.420.05–1 | How much of the schedule to run. 0.20–0.30: very light — just colour/grain matching. 0.30–0.45: typical harmonisation — lighting + soft contact shadows. 0.45–0.60: stronger relight, may shift small details. 0.70+: nearly full re-render — use only if structure is locked by CN. |
| sigma_scheduleopt | COMBO | linear | Step distribution within the active denoise range. balanced is safest. |
| cn_strengthopt | FLOAT | 1.00–5 | ControlNet conditioning scale. 1.0 standard. >1.5 over-constrains. |
| cn_startopt | FLOAT | 0.000–1 | Fraction of steps at which CN guidance begins. |
| cn_endopt | FLOAT | 0.700–1 | Fraction of steps at which CN guidance ends. Stopping at 0.8 lets the last 20%% of steps refine textures without CN over-constraining detail. |
| negative_promptopt | STRING | 低分辨率，低画质，肢体畸形，手指畸形，画面过饱和，蜡像感，人脸无细节，过度光滑，画面具有AI感。构图混乱。文字模糊，扭曲。 | What to avoid (default = official Qwen-Image negative). |
| stepsopt | INT | 404–100 | Total inference steps. 20–28 is a good range; effective denoise steps = round(steps * denoise). |
| true_cfg_scaleopt | FLOAT | 4.51–12 | True CFG. 2.5–3.5 is best for harmonisation. |
| max_sequence_lengthopt | INT | 1024128–1024 | Max prompt token length for the text encoder. 1024 is the Qwen-Image max and has negligible runtime cost — lets long override prompts (per-subject preservation lists) survive without truncation. |
| max_mpopt | FLOAT | 6.00.5–16 | Max megapixels at which to run the harmonisation. The composite is resized to fit; output is upsampled back to composite size with Lanczos. Qwen-Image natively handles up to ~16 MP. Lower this (e.g. 4–8 MP) if you hit VRAM limits. |
| seedopt | INT | 00–18446744073709550000 | Seed (0 = random). |
| upscale_vaeopt | UPSCALE_VAE | Optional. Wire the 2× Wan upscale VAE to replace the final decode with a sharper 2× reconstruction. No extra diffusion is done — the diffusion result is decoded through the upscale VAE instead of the pipeline's standard VAE, then resized down to the composite size with Lanczos. Adds ~1–3s and gives crisper textures and edges. If unwired, the standard pipeline VAE decode is used. | |
| upscale_vae_keep_2xopt | BOOLEAN | true | When upscale_vae is wired and this is ON, output is kept at 2× the working resolution (NOT downsampled to the original composite size). Useful when you want the harmoniser to also act as a 2× upscaler. Ignored if upscale_vae is not wired. |
| refine_passopt | BOOLEAN | true | Run a second, lighter diffusion pass on the result of pass 1. The pass-1 latent stays on the GPU — NO VAE re-encode round-trip — so detail is not lost between passes. Pass 1 = structural harmonisation (lighting, shadows, water blends). Pass 2 = detail refinement (faces, fabric, marble texture). Two gentle passes consistently beat one heavier pass. |
| refine_denoiseopt | FLOAT | 0.300.05–0.6 | Pass-2 denoise. Keep low (0.18–0.28). Pass 1 already locked structure; pass 2 should only sharpen detail. |
| refine_stepsopt | INT | 404–80 | Pass-2 total steps. Effective = round(refine_steps * refine_denoise). |
| refine_sigma_scheduleopt | COMBO | karras | Pass-2 sigma schedule. balanced is best for detail formation. |
| refine_cfgopt | FLOAT | 4.51–12 | Pass-2 CFG. Slightly higher than pass 1 — commits harder to detail words like 'fine skin pores', 'individual eyelashes', etc. |
| refine_cn_strengthopt | FLOAT | 1.00–5 | Pass-2 ControlNet strength. Lower than pass 1 — structure is already locked, so heavy CN here just over-constrains texture. |
| refine_cn_startopt | FLOAT | 0.000–1 | Pass-2 CN start fraction. |
| refine_cn_endopt | FLOAT | 0.500–1 | Pass-2 CN end fraction. Stop CN early so the last 50%% of pass-2 steps are free to refine textures without CN over-constraining. |
| refine_recannyopt | BOOLEAN | false | When ON and cn_mode is auto_canny/auto_soft_edge, re-derive the control image from the pass-1 result (now clean photographic edges, not collage edges). Costs one extra decode-to-PIL but does NOT round-trip the latent. When OFF (or cn_mode = external), pass 2 reuses the pass-1 control image. |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| image | IMAGE | — |
| control_image_used | IMAGE | — |