ComfyUI-RH-Qwen-Image-2-1-diffusers

ComfyUI-RH-Qwen-Image-2-1-diffusers

A ComfyUI wrapper for Qwen-Image 2.1 built on the official huggingface/diffusers pipeline (QwenImage21Pipeline). One node covers all four official modes: text-to-image, single-image editing, multi-reference composition (up to 10 reference images) and RGBA transparency generation.

✨ Features

• One node, four official modes — text-to-image (no image inputs), single-image editing, multi-reference composition with up to 10 reference images (a flat set of condition images shared by the prompt, matching the upstream pipeline contract; later blocks attend to earlier ones, so image order matters), and RGBA transparency generation.
• Official pipeline only — a thin wrapper around QwenImage21Pipeline from diffusers PR #14804: no HTTP serving, no re-implemented network. The three Qwen-Image 2.1 classes are vendored verbatim (Apache-2.0 headers kept, source provenance inside each file); everything else comes from your installed diffusers.
• ComfyUI DynamicVRAM — the three heavy components (Qwen3-VL text encoder, single-stream block-causal transformer, VAE) stream on and off through ModelPatcher + comfy.model_management.load_models_gpu per stage: weights stay on the offload device between runs, so a 24 GiB card runs the full stack and other ComfyUI models stay evictable.
• Official sampling behavior — 40 steps, no guidance (true_cfg_scale=1.0, official recommendation); classifier-free guidance activates only with a non-empty negative_prompt and doubles per-step compute.
• RGBA-capable output — the VAE reconstructs 4 channels; the images output composites alpha over white for the 3-channel ComfyUI IMAGE contract and the alpha MASK output carries the real transparency. Dimensions are rounded to multiples of 32; explicit width/height go up to 2752 for the official 2K ratios (16:9 = 2752x1536).
• Automatic weight discovery — resolves models/diffusers/Qwen-Image-2.1 (case/separator tolerant) or the RH_QWEN_IMAGE21_MODEL_DIR override; the scan lists one directory level and never stats files recursively, so shared NFS model roots stay cheap.

🛠️ Installation

cd ComfyUI/custom_nodes
git clone https://github.com/RH-RunningHub/ComfyUI-RH-Qwen-Image-2-1-diffusers.git
cd ComfyUI-RH-Qwen-Image-2-1-diffusers
pip install -r requirements.txt

torch/torchvision and the CUDA/NVIDIA stack are deliberately not pinned here — the plugin reuses the ComfyUI environment's own torch.

Requirements: ComfyUI with the comfy_api V3 surface (RH fork / ComfyUI ≥ 0.3.x), diffusers>=0.37, transformers>=4.57 (Qwen3-VL), accelerate, and CPU RAM large enough to hold the bf16 pipeline (≈ model size + working set).

📦 Model Download & Installation ⭐

Model directory structure

All model files must be placed under ComfyUI/models/diffusers/Qwen-Image-2.1/ in the standard diffusers layout:

ComfyUI/
└── models/
    └── diffusers/
        └── Qwen-Image-2.1/          # ≈ 33 GB (bf16)
            ├── model_index.json
            ├── scheduler/            # FlowMatchEulerDiscreteScheduler
            ├── text_encoder/         # Qwen3-VL (Qwen3VLForConditionalGeneration)
            ├── processor/            # Qwen3VLProcessor (carries the tokenizer)
            ├── transformer/          # QwenImage21Transformer2DModel
            └── vae/                  # AutoencoderKLQwenImage21

There is no separate tokenizer/ folder in the official repository: the processor component embeds the tokenizer, and the pipeline only needs processor/.

Download methods

Method 1: Download from HuggingFace (Recommended)

cd ComfyUI/models
hf download Qwen/Qwen-Image-2.1 --local-dir diffusers/Qwen-Image-2.1

Method 2: Download from ModelScope (For China users)

pip install modelscope
modelscope download --model Qwen/Qwen-Image-2.1 \
  --local_dir ComfyUI/models/diffusers/Qwen-Image-2.1

Method 3: Manual download

| Component | Link | Description | |-------|------|-------------| | Full pipeline | huggingface.co/Qwen/Qwen-Image-2.1 | All files above, ≈ 33 GB bf16 | | Full pipeline (CN mirror) | modelscope.cn/models/Qwen/Qwen-Image-2.1 | Same content, China-friendly |

When downloading manually, keep the directory layout exactly as shown above (model_index.json plus the component folders).

Placement notes

The node resolves weights automatically from models/diffusers/Qwen-Image-2.1. For a non-standard location, set the environment variable RH_QWEN_IMAGE21_MODEL_DIR to the absolute path of a diffusers-layout directory (a directory containing model_index.json).

🚀 Usage

The examples/ workflows are API-format graphs using the registered node id RH_QwenImage21Diffusers:

• examples/RH_QwenImage21Diffusers_t2i_basic_api.json — text-to-image minimal run.
• examples/RH_QwenImage21Diffusers_i2i_condition_api.json — single-image editing (replace the LoadImage placeholder example.png with a real image first).
• examples/RH_QwenImage21Diffusers_multiref_api.json — multi-reference composition (two LoadImage slots wired to image_1 / image_2; up to ten slots).

For RGBA transparency, use the official prompt format, e.g.: This is an RGBA image with transparency. A cute cartoon dragon sticker. The image has alpha channel and the background is transparent. — images composites the result over white for preview, and alpha carries the real transparency.

Import them into ComfyUI (or submit them through the /prompt API) and press run. A per-run log line reports the resolved model directory, the memory mode actually used, steps, size, condition count, and timings.

📝 Node Reference

Node: RH_QwenImage21Diffusers

| Input | Type | Meaning | |---|---|---| | prompt | STRING (multiline) | Generation prompt; wrapped in the official chat template. | | image_1 … image_10 | IMAGE (optional) | Condition image slots. All empty = text-to-image; 1 = single-image editing; up to 10 = official multi-reference composition (a batched input contributes all its frames in order). | | negative_prompt | STRING (optional) | Only used with true_cfg_scale > 1. | | true_cfg_scale | FLOAT (1.0–10.0) | CFG scale; 1.0 = off (default, official). | | num_inference_steps | INT | Denoising steps; official default 40. | | output_resolution | INT (256–2048) | Target side length; drives both the t2i size and the condition-image resize. | | width / height | INT (0 = auto) | Explicit size override; 0 derives from output_resolution / condition aspect ratio. Max 2752 (official 2K 16:9 = 2752x1536). | | seed | INT | Deterministic generation with the same settings. | | use_kv_cache | BOOL | Official prefix KV cache (default on; causal_condition makes it valid). |

| Output | Meaning | |---|---| | images | ComfyUI IMAGE tensor [B,H,W,3] (alpha composited over white). | | alpha | ComfyUI MASK tensor [B,H,W] (1 = opaque): carries the transparency of RGBA generations; fully opaque for RGB results. |

memory_mode values (reported in the per-run log line):

• comfy-dynamic — the components were loaded through ComfyUI ModelPatcher and the runtime exposes a dynamic patcher (AIMDO per-layer weight streaming). This is the intended mode.
• comfy-full — ModelPatcher staging without the dynamic patcher (weights move as whole modules per stage; still ComfyUI-accounted).
• auto_cpu_offload — fallback: sequential move-to-device staging when the ComfyUI patcher could not be used at all.
• cpu — CPU-only execution (tiny/test environments).

DynamicVRAM details

• Every heavy component is registered with a ModelPatcher (load_device = ComfyUI compute device, offload_device = unet_offload_device()), and each stage calls comfy.model_management.load_models_gpu([patcher], memory_required=…) with an activation reserve (text ~2 GiB, transformer 3–8 GiB scaled by joint-sequence length, VAE decode scaled by output area).
• The vendored transformer/VAE build their Linear/LayerNorm/Conv2d layers through comfy.ops.disable_weight_init subclasses (qwen_image21_nodes/compat_ops.py), which is what lets AIMDO stream weights per layer under memory pressure.
• On load OOM: unload_all_models() + soft_empty_cache() and one retry with a doubled reserve; afterwards the driver degrades to auto_cpu_offload staging automatically (no user-facing dropdown).
• Text and condition images are encoded with the text encoder resident; the transformer never co-resides with the text encoder, mirroring the upstream model_cpu_offload_seq.

📄 License

• Plugin code and the vendored diffusers classes: Apache-2.0 (see LICENSE and NOTICE).
• Model weights Qwen/Qwen-Image-2.1: distributed under the Qwen Research License Agreement (qwen-research) — check the official model card before commercial use.

🔗 Links

• Upstream pipeline: huggingface/diffusers PR #14804 — Add Qwen-Image 2.1
• Model page: Qwen/Qwen-Image-2.1 (also on ModelScope)
• Qwen-Image (2.0 lineage) model page: Qwen/Qwen-Image
• diffusers: github.com/huggingface/diffusers

🙏 Acknowledgements

This project wraps the official QwenImage21Pipeline contributed to huggingface/diffusers, built on the Qwen-Image 2.1 model developed by the Qwen team (QwenLM).

Local tests

HF_ENDPOINT=https://hf-mirror.com pytest -q tests/test_tiny_smoke.py