Nodes/ComfyUI_Pops/Pops_Repo_Loader
ComfyUI Node

Pops_Repo_Loader

The pOps operator stack

By smthemex·Created 2 years ago·Updated 2 years ago· 21
Pops_Repo_Loader
    • model
    • clip
    • vae
    • tokenizer
    local_prior
    prior_repokandinsky-community/kandinsky-2-2-prior
    local_decoder
    decoder_repokandinsky-community/kandinsky-2-2-decoder
    pops_ckpt
    function_type

    The name makes it sound like a boring file picker. It's not - Pops_Repo_Loader is where the whole ComfyUI_Pops project comes to life, and it's the node that decides what "operator" you're running. Wire it up, and the other two nodes in the pack (Pops_Sampler and Pops_Decode) suddenly mean something. Skip it, and nothing in the pack works, because everything else is just moving embeddings around.

    What pOps actually is

    The paper behind this is pOps: Photo-Inspired Diffusion Operators (Richardson et al., 2024). The idea: instead of describing a transformation in words, you train a small diffusion model to operate directly on CLIP image embeddings. The "operators" this produced are things like:

    • texture - give it an object picture + a texture picture, and it paints the texture onto the object
    • scene - drop your object into a semantically similar scene
    • union - merge the objects from two pictures into one
    • instruct - one object image + a single adjective ("smooth", "wooden") and it applies that quality

    That image-embedding trick is the same territory IP-Adapter popularized for style transfer, except pOps trains the operation itself rather than injecting a reference. Each operator is a fine-tuned Diffusion Prior - the component from the Kandinsky 2.2 text-to-image pipeline that normally maps text embeddings to image embeddings. pOps retrained it to map between image embeddings instead. The actual pixels are then drawn by Kandinsky 2.2's decoder, which is why this pack depends on Kandinsky models you probably don't have yet.

    What this node does

    Pops_Repo_Loader loads three pieces into memory and hands them to you as outputs:

    1. The pOps pipeline - the base Kandinsky 2.2 prior with one of the four trained operator checkpoints fused on top (model)
    2. A CLIP image processor for turning your input images into embeddings (clip)
    3. The Kandinsky 2.2 decoder pipeline, labeled as a VAE (vae)
    4. A tokenizer for the instruct operator - only when you ask for it (tokenizer)

    One gotcha worth knowing up front: this "clip" is a CLIPImageProcessor, not the text-CLIP you'd normally feed a CLIPTextEncode. Don't cross-wire these outputs into regular ComfyUI nodes. Everything here is a matched set that only the pack's own Sampler and Decoder understand.

    The inputs that matter

    • pops_ckpt - the operator checkpoint, picked from your ComfyUI/models/checkpoints folder. The four files are texturing_learned_prior.pth, scene_learned_prior.pth, union_learned_prior.pth, instruct_learned_prior.pth (renamed from the pOpsPaper/operators repo layout). This is the whole point of the node.
    • function_type - Binary for the two-image operators (texture/scene/union), instruct for the text-guided one. It must match the checkpoint you loaded: instruct loads the tokenizer and unlocks the text input on the Sampler; Binary doesn't.
    • prior_repo / decoder_repo - HuggingFace repo IDs, defaulting to kandinsky-community/kandinsky-2-2-prior and kandinsky-2-2-decoder.
    • local_prior / local_decoder - pick a locally downloaded diffusers folder instead of a repo ID, for offline use.

    Installation

    ComfyUI Manager is the easy path - search for ComfyUI_Pops and hit install. Manually:

    cd ComfyUI/custom_nodes
    git clone https://github.com/smthemex/ComfyUI_Pops.git
    cd ComfyUI_Pops
    pip install -r requirements.txt
    

    Restart ComfyUI after that. Be aware the requirements.txt only lists pyrallis - the code really needs diffusers and transformers (it imports PriorTransformer, UNet2DConditionModel, KandinskyV22Pipeline and several CLIP pieces). If you already run any modern nodes you'll have those; if not, install them or the node will throw import errors on load.

    The heavy part: models

    Leave the repo fields at their defaults and the loader will auto-download the Kandinsky 2.2 prior and decoder from HuggingFace on first use - that's a couple of gigabytes. The operator checkpoints come from pOpsPaper/operators. For offline use, the README's layout is:

    ComfyUI/models/diffusers/kandinsky-community/kandinsky-2-2-prior/...   (full diffusers folder)
    ComfyUI/models/diffusers/kandinsky-community/kandinsky-2-2-decoder/... (full diffusers folder)
    ComfyUI/models/checkpoints/<operator>_learned_prior.pth                (renamed ckpts)
    

    Where people get burned

    The most common failures are: loading an instruct checkpoint while leaving function_type on Binary (the text path on the Sampler silently won't work), picking none for pops_ckpt and getting the untrained base prior, and running out of VRAM - this node holds the prior, image encoder, and the whole decoder in memory at once, in float16, all on CUDA. It's also worth knowing this pack is a research port with a tiny user base; the README itself notes the SDXL section was removed and quality is "抽卡的几率太大" - highly variable. If you're chasing a specific production effect, temper expectations. The author, smthemex, is a prolific paper-porting node author (TwinFlow, AnyDoor, ParlerTTS, and dozens more), and this one is closer to a reproduction than a polished tool.

    CategoryPops

    Inputs (6)

    NameTypeDefaultDescription
    local_priorCOMBO1 options: none
    prior_repoSTRINGkandinsky-community/kandinsky-2-2-prior
    local_decoderCOMBO1 options: none
    decoder_repoSTRINGkandinsky-community/kandinsky-2-2-decoder
    pops_ckptCOMBO1 options: none
    function_typeCOMBO2 options: Binary, instruct

    Outputs (4)

    NameTypeDescription
    modelMODEL
    clipCLIP
    vaeVAE
    tokenizerMODEL