Nodes/ComfyUI-J/🤗 Diffusers Pipeline
ComfyUI Node

🤗 Diffusers Pipeline

The checkpoint loader that skips ComfyUI's own sampler stack entirely

By Jannchie·Created 2 years ago·Updated about a year ago· 98
🤗 Diffusers Pipeline
    • pipeline
    ◄ckpt_name▾►
    â—„vae_name-â–º
    â—„scheduler_name-â–º
    â—„use_tiny_vaedisableâ–º

    This is the entry point for a whole parallel universe inside ComfyUI. Every other Jannchie node in this pack - the prompt embedder, the ControlNet units, the reference-only sampler - plugs into what this node hands out, and none of it talks to ComfyUI's native MODEL/CLIP/VAE trio at all. Once you drop this node, you're committing to Diffusers-flavored nodes for the rest of that branch of the graph.

    Why this exists instead of just using CheckpointLoaderSimple

    The pack's own README is refreshingly blunt about the motive: the author got tired of ComfyUI's reference-only support being incomplete, found inpainting behaving unlike he expected, couldn't figure out why ControlNet nodes needed a CLIP input, and didn't want to manually VAE-decode every latent. HuggingFace's Diffusers library already solves all of that with a pipeline object that does prompting, sampling, and decoding as one unit and just hands you back an image. So instead of fighting ComfyUI's native implementation, this pack wraps Diffusers directly and exposes it as nodes. It's a genuinely different philosophy from the rest of the ecosystem - most custom nodes extend ComfyUI's SD implementation, this one replaces it.

    That's also the honest trade-off: you're now running two independent implementations of Stable Diffusion side by side on the same machine (ComfyUI's own, plus Diffusers), which is more disk, more Python dependencies, and no way to swap a stock KSampler into the middle of a ComfyUI-J chain.

    What it actually does

    Given a checkpoint name, it builds a Diffusers StableDiffusionPipeline-style object - text encoder, UNet, scheduler, and VAE all bundled - and hands it out as a single DIFFUSERS_PIPELINE object. Everything downstream (prompt embedding, ControlNet units, the generator node) takes that pipeline as an input rather than separate model/clip/vae connections. It's one wire instead of three.

    The inputs and outputs that matter

    • ckpt_name (required) - reads from the same checkpoints folder ComfyUI already uses. No separate download step for your base model; if it shows up in ComfyUI's own checkpoint loader, it'll show up here.
    • vae_name (optional, default -) - override the checkpoint's baked-in VAE with a standalone one. Leave it at - unless you have a specific reason to swap; a mismatched VAE gets you the classic washed-out or noisy output rather than anything better.
    • scheduler_name (optional, default -) - 18 choices, the familiar DPM++ 2M/SDE/Karras and Euler family. Since this pack only targets SD1.5 and SDXL checkpoints, the old "DPM++ 2M Karras is the safe default" advice applies cleanly here - you don't need to worry about the flow-matching caveats that trip people up on Flux or Z-Image. Leaving it at - uses whatever the checkpoint's own config specifies.
    • use_tiny_vae (optional, default disable) - swaps in a distilled, much faster approximate VAE for decoding, the same trade ComfyUI's own TAESD preview makes: quicker, lower fidelity. Handy while you're iterating on a prompt; turn it off for a final pass.

    Output is a single pipeline (DIFFUSERS_PIPELINE) - wire it into the Compel prompt node, the ControlNet loader chain, or straight into the Generator.

    Installing it

    ComfyUI Manager: search ComfyUI-J, install, restart. Or by hand:

    cd ComfyUI/custom_nodes
    git clone https://github.com/Jannchie/ComfyUI-J
    cd ComfyUI-J
    pip install -r requirements.txt
    

    That pip install step matters more than it does for most packs - it pulls in the Diffusers library itself, which is a second, independent implementation of the sampling stack living in the same Python environment as everything else you've installed. ComfyUI gives every custom node's dependencies the same unshared environment with no isolation, so if you've got a dozen other packs installed, this is exactly the kind of install that can quietly bump transformers or accelerate to a version something else pinned against. If things break after adding this pack, check for a version conflict before assuming the node itself is broken.

    Where people get burned

    The big one: no LoRA support. It's on the pack's own TODO list and still unchecked as of this writing, so if your usual workflow leans on a character or style LoRA, there's no slot for it anywhere in this chain - you'd need to fall back to ComfyUI's native loader for that piece. Second, don't expect to mix ComfyUI-J nodes with stock ComfyUI mid-graph; the DIFFUSERS_PIPELINE type has no equivalent on the native side, so it's an all-or-nothing branch. And this is a niche pack - it doesn't show up in mainstream showcase workflows, but it's real: it's the node pack behind at least one widely-shared 2024 compact inpainting workflow, credited by name in the post because four Diffusers nodes replaced what would've been seven stock ones for the same result.

    CategoryJannchie

    Inputs (4)

    NameTypeDefaultDescription
    ckpt_nameCOMBO0 options:
    vae_nameoptCOMBO-1 options: -
    scheduler_nameoptCOMBO-18 options: DPM++ 2M, DPM++ 2M Karras, DPM++ 2M SDE, DPM++ 2M SDE Karras, DPM++ SDE, DPM++ SDE Karras, +12
    use_tiny_vaeoptCOMBOdisable2 options: disable, enable

    Outputs (1)

    NameTypeDescription
    pipelineDIFFUSERS_PIPELINE—