SD3 Image Into Attention

This is the node that justifies the pack's name. SD3 Image Into Attention takes a picture, flattens it, and writes it into the attention weights of your model - then hands you back a patched model that will generate differently because its internals just changed. No fine-tuning, no LoRA, no training run. Point-blank weight surgery, live in the graph.

It's the write-back half of the pair that starts with SD3 Attention To Image. That node dumps a block's QKV weight matrix as an exact 1536×1536 RGB image; this one reverses it - .view(4608, 1536) on the image - and installs it into whichever joint block and backbone you name. Because an image is just an array of numbers, and weights are just an array of numbers, the two talk to each other perfectly. The pack came out of the June 2024 "perturbed SD3" experiments, when the community was trying to reverse-engineer what Stability's safety training had done to the model; the author built these nodes specifically so people could point-blank replace attention layers and see what broke.

The inputs that matter

• sd3_model - the checkpoint to patch. The node clones it first, so your original stays untouched.
• joint_block - which of the 24 blocks (0–23).
• backbone - text (context_block) or latent (x_block).
• attention_image - must be exactly 1536×1536 pixels. Its three channels become the three 1536-row chunks of the 4608×1536 QKV matrix. Any other size will throw a reshape error.
• patch_strength (default 1.0) and model_strength (default 0.0) - the mix knobs.

patch_strength vs model_strength

This is the one thing worth understanding before you use it. These map to ComfyUI's add_patches machinery - the same mechanism LoRAs ride on - and they control the blend between your image and the original weights. With the defaults (1.0 / 0.0) your image replaces the layer completely: full patch, zero original. To keep more of the original model in the mix, lower patch_strength (how much of your image is added) or raise model_strength (how much of the original is kept). If you dump a block, feed the untouched dump back at 1.0/0.0, you get the original model back. Adjust the strengths and you're interpolating between stock and your edit - which is how you actually experiment without nuking a layer.

The output is a single MODEL, wired straight into your sampler in place of the original.

Installing it

ComfyUI Manager → search ComfyUI-SD3-Powerlab → install → restart, or:

cd ComfyUI/custom_nodes
git clone https://github.com/G-370/ComfyUI-SD3-Powerlab

No requirements.txt and no model files to fetch. The one environment trap: the pack imports matplotlib at load time, which ComfyUI doesn't ship - if the nodes never appear, pip install matplotlib into ComfyUI's Python.

Where people get burned

The 1536 dimension is hardcoded to SD3 Medium's architecture, so the reshape breaks on SD 3.5 Large or Medium. And two less obvious traps: the image must be exactly 1536×1536 or the flattening silently reshapes into the wrong shape, and if the block/backbone name doesn't resolve, the error is raised as a bare string - Python turns that into a baffling TypeError: exceptions must derive from BaseException instead of the message the author wrote. Also brace yourself: this is an experimental toy from a summer-2024 research sprint, not a production technique. It's genuinely fun to see a model's behavior change because you painted over a weight matrix - but treat every generation as a hypothesis test, and keep your base checkpoint safe, because the things you can do here are mostly how you learn what a layer does.