ComfyUI Node

LCM_IPAdapter

Image-prompting on the fast LCM pipeline

By taabata·Created 3 years ago·Updated 2 years ago· 259
LCM_IPAdapter
    • class
    device
    ip_adapter
    ip_adapter_full_path

    LCM_IPAdapter is the loader that makes this pack's "image variations" and "style transfer" tricks work. It builds the LCM_Dreamshaper_v7 pipeline and loads an IP-Adapter into it, so you can prompt by example - hand it a reference picture and the model steers toward that style or subject without training anything. If you're here from the pack's Style Transfer or Image Variations screenshots, this is the node behind them.

    How it works

    IP-Adapter is a ~22M-parameter adapter that injects an image embedding through its own cross-attention path, leaving the text path alone - that's the whole trick that made "prompt with a picture" cheap. This loader does three things at once: builds the LCM pipeline, picks an IP-Adapter checkpoint, and locates a CLIP vision encoder to embed the reference. The inputs:

    • device - cpu or cuda. Note the ordering is lowercase here, unlike the loaders' GPU/CPU.
    • ip_adapter - dropdown of files in ComfyUI/models/controlnet ending in .ckpt, .safetensors, or .bin. Here's a fun detail from the source: if the filename contains plus, it loads IPAdapterPlus with 16 tokens (the stronger, more recent adapter); otherwise plain IPAdapter. So the dropdown filename actually changes the mechanism.
    • ip_adapter_full_path - leave blank to use the dropdown; fill it in to point at a checkpoint anywhere else on disk. Useful when the dropdown misses your file.

    The output class is an IP-Adapter wrapper around the LCM pipe, and it feeds the ip_model input on LCMGenerate_img2img_IPAdapter (and the ip_model/pil_image/scale inputs on LCMGenerate_inpaintv3).

    Setup

    The pack install is the same everywhere - ComfyUI Manager (search LCM_Inpaint-Outpaint_Comfy) or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/taabata/LCM_Inpaint-Outpaint_Comfy
    cd LCM_Inpaint-Outpaint_Comfy
    pip install -r requirements.txt
    

    Then three model pieces: LCM_Dreamshaper_v7 in ComfyUI/models/diffusers/, the IP-Adapter .safetensors in ComfyUI/models/controlnet/, and a CLIP vision encoder in ComfyUI/models/clip_vision/ - the loader reads that folder for the image encoder. Miss any of the three and the loader dies with a missing-file or "couldn't connect to huggingface.co" style error.

    Gotchas

    The device arg is basically cosmetic here - the IP-Adapter code path comments out the .to("cuda")/.to("cpu") calls, so don't expect cpu to give you a working CPU path. The real risk is VRAM: you've got the diffusers LCM model plus an IP-Adapter loaded on top of ComfyUI's own checkpoint cache, and on 8 GB cards this node is where things start getting tight.

    And the honest caveat: this is late-2023 code, before IP-Adapter really matured in ComfyUI. The idea is solid and the results are genuinely fun - style transfer on a 4-step model is almost real-time. But today's native IPAdapterApply nodes are better documented and better maintained. Reach for this node when you specifically want the pack's single-node LCM+IPAdapter pipeline.

    CategoryLCM_Nodes/nodes

    Inputs (3)

    NameTypeDefaultDescription
    deviceCOMBO2 options: cpu, cuda
    ip_adapterCOMBO0 options:
    ip_adapter_full_pathSTRING

    Outputs (1)

    NameTypeDescription
    classclass