Apply Instance Diffusion ScaleU

ApplyScaleUModelNode ("Apply Instance Diffusion ScaleU") is the least exciting node in the pack and the one you'll forget to add. Then your first multi-instance image comes out soft, edges blurry, details smeared, and you'll remember. It's the fidelity stage of InstanceDiffusion: the piece that takes an otherwise-correct image and makes it look finished instead of like a first draft.

The Context, real quick: InstanceDiffusion is the Berkeley research project (arXiv 2402.03290) that lets you give each subject in an image its own prompt and its own tracked location. This ComfyUI port by logtd splices the trained weights into three components - positionnet, fusers, and ScaleU - so you can bolt them onto any SD1.5 checkpoint. ApplyScaleUModelNode is the third of those three, and it does one job: improve image fidelity.

How it works

In the original paper, ScaleU is one of the three contributions that made instance-level control actually look good (the others being UniFusion, which is the positionnet, and the multi-instance fuser sampler). The ComfyUI version registers what ComfyUI calls an output_block_patch on your model. During sampling, each of the twelve UNet output blocks gets pushed through a ScaleU module that does two things: an SE-style channel rescaling that emphasizes the important feature channels, and a Fourier-domain low-pass filter that scrubs high-frequency noise. That combination is why output with the node applied reads as cleaner and sharper than without.

Inputs and outputs

Only two inputs, both required - this is the simplest node in the pack, and that's a feature.

• model (MODEL) - the SD1.5 model from your checkpoint loader. Wire the MODEL output here before it hits the KSampler.
• scaleu (SCALEU) - the loaded ScaleU weights, from the DownloadInstanceDiffusionModels node or the individual LoadInstanceScaleUNode.

Output is a single patched MODEL, which goes into the same KSampler that receives the positive/negative conditioning from InstanceDiffusionTrackingPrompt. That wiring order matters: the patch only takes effect if the patched model is what you feed the sampler.

Installing it

Same install as the whole pack, since these all ship together: ComfyUI Manager, search "InstanceDiffusion", or clone it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/logtd/ComfyUI-InstanceDiffusion

Then restart ComfyUI. The pack's requirements.txt pins only huggingface_hub, which ComfyUI already ships, so there's no real dependency install to babysit.

Gotchas

• SD1.5 only. The ScaleU weights were trained baked into Stable Diffusion 1.5. Don't wire this into an SDXL or Flux graph; it'll quietly do nothing useful. This whole pack is an SD1.5 play in a world that's mostly moved on - it still wins for tracked multi-subject animation, which is why people put up with the old base model.
• It does nothing without the fusers and positionnet. ScaleU is the polish layer, not the mechanism. If you only add this node, your image will just be a clean image with no instance control. The tracking prompt node is the engine; this is the paint job.
• It won't fix a bad prompt separation. If two subjects are still bleeding attributes into each other, ScaleU isn't the lever - that's the fusers_scale dial on the loader. ScaleU only makes what's already correct crisper.