Nodes/RES4LYF/ClownStyle_SpatialBlock_UNet
ComfyUI Node Runs on cloud

ClownStyle_SpatialBlock_UNet

Style transfer inside a U-Net's spatial transformer

By ClownsharkBatwing·Created 2 years ago·Updated 17 days ago· 1,222
ClownStyle_SpatialBlock_UNet
  • mask
  • blocks
  • blocks
modescattersort
block_typeinput
block_listall
block_weights1.0
norm_in0.00
proj_in0.00
transformer_block0.00
transformer0.00
proj_out0.00
res0.00
tile_h128
tile_w128
invert_maskfalse

This is the middle rung of RES4LYF's U-Net style-transfer control, between ClownStyle_Block_UNet's single coarse spatial tap and ClownStyle_TransformerBlock_UNet's fully granular per-sublayer control. Instead of turning the whole spatial-transformer submodule up or down as one unit, it opens that submodule up into its own six parts. Chain it into ClownStyle_UNet's blocks input like the rest of the family, for style-transfer work on SD1.5, SDXL, or other classic U-Net models.

How it works

A U-Net's SpatialTransformer submodule - the attention block bolted onto each ResBlock - is built as: a normalization before entry, a projection into transformer token space, one or more transformer blocks doing the actual attention work, and a projection back out to the convolutional feature map, plus the residual connection tying it back to the ResBlock's output. This node exposes each of those stages as its own float: norm_in (the entry normalization), proj_in (the projection into token space), transformer_block and transformer (two related taps on the attention work itself, at slightly different points in how it's applied - if you're not sure which to reach for, try transformer first since it's the broader of the two), proj_out (the projection back out), and res (the residual add). All six default to 0.

The inputs and outputs that matter

  • mode - the style-matching algorithm (scattersort, AdaIN, WCT, and the rest - shared with ClownStyle_UNet).
  • block_type (input / middle / output, or combinations) - which U-Net stage(s) to target.
  • block_list (default "all") / block_weights (default "1.0") - narrow to specific blocks and weight them.
  • norm_in, proj_in, transformer_block, transformer, proj_out, res - the six taps described above, all default 0 (off).
  • mask + invert_mask restrict spatially; tile_h/tile_w only matter for tiled_ modes.
  • blocks (optional in/out) chains this into the rest of the ClownStyle_* family, ultimately into ClownStyle_UNet.

How to install it

Via ComfyUI Manager, search "RES4LYF". Or:

cd ComfyUI/custom_nodes
git clone https://github.com/ClownsharkBatwing/RES4LYF/
cd RES4LYF
pip install -r requirements.txt

then restart.

Common issues & troubleshooting

This is a lot of granularity for most use cases - you probably don't need it first. Try ClownStyle_Block_UNet's single spatial tap or ClownStyle_UNet's own proj_in/proj_out before reaching this deep; only come here once you know you want control at this specific level and the coarser nodes weren't precise enough.

transformer_block and transformer will often look similar when you toggle them individually since they're both taps on the same underlying attention work at nearby points - if you're chasing a specific effect, it's worth A/B-ing the two rather than assuming which one you need.

All six taps default to off, same as everywhere else in this family. No visible effect after wiring this in means no tap is turned on yet, not that the node failed to apply.

CategoryRES4LYF/sampler_extensions

Inputs (15)

NameTypeDefaultDescription
modeCOMBOscattersort10 options: none, scattersort_dir, scattersort_dir2, scattersort, tiled_scattersort, AdaIN, +4
block_typeCOMBOinput7 options: input, middle, output, input,middle, input,output, middle,output, +1
block_listSTRINGall
block_weightsSTRING1.0
norm_inFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
proj_inFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
transformer_blockFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
transformerFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
proj_outFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
resFLOAT0.00-100–100Strength of effect on layer; skips extra calculation if set to 0.0. Skips interpolation if set to 1.0.
tile_hINT12816–10000Tile size for tiled modes. Lower values will transfer composition more effectively. Dimensions of image must be divisible by this value.
tile_wINT12816–10000Tile size for tiled modes. Lower values will transfer composition more effectively. Dimensions of image must be divisible by this value.
invert_maskBOOLEANfalse
maskoptMASK
blocksoptBLOCKS

Outputs (1)

NameTypeDescription
blocksBLOCKS