(Down)Load Instance Diffusion Models

DownloadInstanceDiffusionModels ("(Down)Load Instance Diffusion Models") is the pack's setup node - the one that turns "install this pack" from a three-file scavenger hunt into a single node drop. InstanceDiffusion needs three small trained components (positionnet, fusers, ScaleU) that the author spliced out of the original SD1.5-baked research weights. This node downloads all three and loads them in one go.

The README's manual path tells you to fetch three .ckpt files and place them into subfolders of ComfyUI/models/instance_models/ by hand. This node does exactly that for you: it checks each location, downloads whatever's missing from the HuggingFace repo logtd/instance_diffusion, and returns the loaded weights as the three custom types the other nodes want. It's also the nice thing to include in a shared workflow, because anyone who loads it gets the files pulled automatically on their first run.

Inputs

Just two, both optional-looking but technically required:

• use_segs (BOOLEAN, default true) - whether the positionnet conditions on segmentation masks from the tracker rather than plain bounding boxes. Leave it on; your tracking source (typically a YOLO tracker) produces segs, and it gives the positionnet more to work with.
• fusers_scale (FLOAT, 0–10, default 1.0) - the strength of the per-instance attention fusers. This is your main "make the subjects actually follow their prompts" dial. Start at 1.0; if two instances keep swapping attributes or drifting, nudge it up.

Outputs

Three, one per component, and they map cleanly onto the rest of the pack:

• POSITIONNET → the InstanceDiffusionTrackingPrompt node
• FUSERS → the InstanceDiffusionTrackingPrompt node
• SCALEU → the ApplyScaleUModelNode

So the wiring is: this node feeds everything, the tracking prompt node does the per-instance conditioning, ApplyScaleUModelNode polishes fidelity, all into one KSampler. You never touch the three individual loaders (LoadInstanceScaleUNode, LoadInstancePositionNetNode, LoadInstanceFusersNode) unless you want per-file control instead of the bundle.

Installing

Same pack as everything else: ComfyUI Manager → search "InstanceDiffusion", or:

cd ComfyUI/custom_nodes
git clone https://github.com/logtd/ComfyUI-InstanceDiffusion

Then restart. The pack's only Python requirement is huggingface_hub, which ComfyUI already bundles, so installation is genuinely light - the heavy part is the model downloads, which this node handles.

Gotchas

• First run needs internet and a few minutes. You're pulling roughly a gigabyte across three files. If it looks hung, it's probably just downloading; check the console.
• Filename exactness. The node checks for files named exactly position_net.ckpt, fusers.ckpt, and scaleu.ckpt. If you hand-placed one with a different name from the README's HuggingFace links, it'll re-download rather than reuse it. Match the names and it skips straight to loading.
• SD1.5 only. These weights are trained into Stable Diffusion 1.5, so pair them with an SD1.5 checkpoint - DreamShaper or similar - not SDXL. The whole pack is a 2024 SD1.5-era tool, still worth using for tracked multi-subject animation, but know what you're loading before you wire it into a modern graph.