ComfyUI Node

InstantIR_Loader

Five files in, one pipe out

By smthemex·Created 2 years ago·Updated about a year ago· 242
InstantIR_Loader
    • model
    sdxl_checkpoints
    dino_repofacebook/dinov2-large
    adapter_checkpoints
    aggregator_checkpoints
    lora
    InstantIR_lora
    use_clip_encoderfalse

    InstantIR doesn't call an API and needs no key, but it's also not one download-and-go model. It's five pieces that only do anything assembled, and InstantIR_Loader is the node that assembles them: an SDXL checkpoint, three InstantIR weights, a DINOv2 vision encoder, and an SDXL LCM LoRA, all fused into one InstantIR_MODEL object you hand to the pack's other node, InstantIR_Sampler.

    So this is the boring node - nothing that affects image quality lives here. Get the file paths right and you'll never touch it again. Get them wrong and you'll stare at errors written in broken English, so let's map it properly.

    What you're actually building

    InstantIR is InstantX's blind image restoration model (arXiv 2410.06551). "Blind" means it takes a photo with unknown damage - blur, compression, noise - and repairs it without being told what's wrong. The degraded image is encoded by a DINO-based module into conditioning injected into an SDXL UNet's cross-attention, in the same spirit as IP-Adapter. At each sampling step a consistency-distilled "previewer" predicts what the clean result should look like, and an "aggregator" fuses that prediction back into the network. The output is generated plausible detail, not sharpened pixels.

    That places it on the generative-restoration rung of the upscaling ladder - the SUPIR/SeedVR2 family, not ESRGAN. It rebuilds rather than recovers. Two honest caveats before you commit disk space: this is late-2024 tech, and by 2026 most people doing this job had moved to SeedVR2; and because it generates, it will rewrite faces. For genuinely damaged photos where you want a free, local, SDXL-based fix, it's still a perfectly reasonable pick.

    The inputs that matter

    Every one of these is mandatory - leave any dropdown on "none" and the node raises. There's no graceful "optional model" path here.

    • sdxl_checkpoints - any SDXL checkpoint from ComfyUI/models/checkpoints. It must be SDXL; the wrapper builds an SDXL pipeline and an SD 1.5 checkpoint will fail to convert.
    • dino_repo - defaults to facebook/dinov2-large, the vision encoder. Leave it as-is; if you blank the field it auto-downloads into ComfyUI/models/InstantIR/dino/.
    • adapter_checkpoints, aggregator_checkpoints, InstantIR_lora - adapter.pt, aggregator.pt, and previewer_lora_weights.bin from InstantX/InstantIR on Hugging Face, dropped in ComfyUI/models/InstantIR/models/. All three are required.
    • lora - the SDXL LCM LoRA (latent-consistency/lcm-lora-sdxl) from your loras folder. Also required - this is the 20x speed trick.
    • use_clip_encoder - false by default, and that's the DINO path. Flip it only if you know you want the CLIP text-conditioned variant. You don't.

    Output is a single model (InstantIR_MODEL), which feeds the sampler's model input. That's the whole loop.

    Install

    Easiest path: ComfyUI Manager → search "ComfyUI_InstantIR_Wrapper" → install. Manual, the README way:

    cd ComfyUI/custom_nodes
    git clone https://github.com/smthemex/ComfyUI_InstantIR_Wrapper.git
    # then, into your ComfyUI Python environment:
    pip install -r requirements.txt
    

    The requirements are heavy and overlap ComfyUI's own env - diffusers, kornia, opencv-python, peft. That's another reason Manager is the nicer route; a bare pip install can clobber versions other nodes rely on.

    Then the downloads: the three InstantIR weights, the LCM LoRA, and DINO (which fetches itself on first run).

    Where people get burned

    • The raise messages. Leave anything unset and you get "need chocie a sdxl checkpoint" - typo and all. It means "pick something from the dropdown."
    • File placement. The loader scans ComfyUI/models/InstantIR for .pt/.bin files. Keep the three weights in the models subfolder like the README says, and they'll appear in the dropdowns.
    • Picking an SD 1.5 checkpoint. Easy mistake if your checkpoints folder is mixed. SDXL only.
    • VRAM. The loader calls enable_model_cpu_offload() unless you're on MPS, which is why the README's November 2024 patch claims ~10 GB peak and ~20 seconds at 20 steps on a 12 GB 4070. If you're crawling or OOMing and it used to be fast, you're probably missing the LCM LoRA.
    CategoryInstantIR

    Inputs (7)

    NameTypeDefaultDescription
    sdxl_checkpointsCOMBO1 options: none
    dino_repoSTRINGfacebook/dinov2-large
    adapter_checkpointsCOMBO1 options: none
    aggregator_checkpointsCOMBO1 options: none
    loraCOMBO1 options: none
    InstantIR_loraCOMBO1 options: none
    use_clip_encoderBOOLEANfalse

    Outputs (1)

    NameTypeDescription
    modelInstantIR_MODEL