Nodes/ComfyUI-productfix/VQ loader diffusers (middlek)
ComfyUI Node

VQ loader diffusers (middlek)

The loader for diffusers VQ models — the niche-of-a-niche latent codec step

By MiddleKD·Created 2 years ago·Updated about a year ago· 21
VQ loader diffusers (middlek)
    • VQ
    vq_name

    The VQ trio (VQLoaderVQEncoderVQDecoder) is the pack's alternative image↔latent path, in the style of Kandinsky 2.2 and MoVQ - a quantized vector autoencoder instead of the usual SD VAE. VQLoader is the front door: it finds a VQ checkpoint on disk and loads it as a diffusers VQModel.

    How it works

    The node scans a very specific folder - ComfyUI's vae_approx directory - for files with "vq" in the filename. That folder normally holds the tiny approximate-VAE preview models ComfyUI uses for latent thumbnails, so it's a slightly odd place to look; the author just repurposed it as "the place VQ checkpoints live." Your dropdown will be empty until you put a file with vq in its name there.

    Loading supports two shapes. If the file is a raw checkpoint (a torch.load dict containing configs and state_dict keys), it builds a VQModel from the config and loads the state dict. If it's a diffusers model folder, it falls back to VQModel.from_pretrained. Either way you get a VQ object back, offloaded to CPU-ish intermediate device and cast to the VAE dtype to keep VRAM free.

    The input and output

    • vq_name - dropdown, auto-populated from ComfyUI/models/vae_approx/. Empty until you add a file.
    • Returns a VQ - wire it into VQEncoder and VQDecoder.

    Where do you get a checkpoint? The README's references point at Kandinsky 2.2 and MoVQ; you'd grab a VQGAN/MoVQ-style weights file, rename it with vq in the name, and drop it into vae_approx. That's the entire setup.

    Install

    cd ComfyUI/custom_nodes
    git clone https://github.com/MiddleKD/ComfyUI-productfix
    pip install -r requirements.txt
    

    This node specifically needs diffusers (in the pack's requirements) and a VQ weights file - it will not download one for you.

    Honest assessment

    This is the most specialized corner of an already niche pack. The VQ path only matters if you're deliberately composing in a quantized latent space in the Kandinsky lineage - the pack's latent-injection approach leans on it, but the shipped demo workflows don't even require it. If you're not building that, you can skip the whole trio and never miss it. And if you are building that, you're in "understand the source or write your own" territory, because there's essentially no community documentation to lean on. It's a working diffusers VQModel wrapper; it does what it says, and nothing more.

    Categoryproductfix

    Inputs (1)

    NameTypeDefaultDescription
    vq_nameCOMBO0 options:

    Outputs (1)

    NameTypeDescription
    VQVQ