HF Diffusers VAE encode

This is the low-level VAE encoder for the HuggingFace side of this pack. Feed it a real image and a HuggingFace VAE (the one HFDAutoencoderKL loads, or the one HFDLoadPipeline hands you as its second output), and it gives you a ComfyUI-format latent you can sample from. It's the img2img bridge: without an encode step, a text-to-image pipeline has no way to start from your picture.

The name is more honest than most: it's just VAE encode. Two inputs, one output, nothing hidden.

• image - a PIL_IMAGE. Note the type. This pack speaks PIL, not ComfyUI's native IMAGE, so the image has to come from a PIL-bridge node (the author recommends Chaoses-Ib's ComfyUI_Ib_CustomNodes, whose ImageToPIL is the reverse of the PILToImage you'll see at the end of these workflows). If you feed it a ComfyUI IMAGE directly it won't connect.
• vae - an HFD_AUTOENCODERKL from the pack's VAE loader.
• output - LATENT, and this one is native ComfyUI, so it plugs straight into a regular KSampler or into HFDRunPipeline's optional latents input.

How it works

Under the hood it's a textbook diffusers encode: VaeImageProcessor preprocesses the PIL image to a tensor, moves it to the VAE's device and dtype, runs vae.encode(), takes the sampled latent from the result, and multiplies by the model's scaling_factor - the same bookkeeping the pipeline would do itself. The VAE's downsampling factor is derived from its block_out_channels config, so it works across architectures rather than assuming 8x.

When you'd actually reach for it

Two honest use cases. One: you want to run img2img through HFDRunPipeline but control the encode yourself, or encode with a different VAE than the pipeline will use. Two: you're mixing worlds - encode an image with a HuggingFace VAE, then sample it with the native ComfyUI sampler, which is a genuinely useful trick when the model you want isn't one ComfyUI's own loaders can read. It's a specialist node; most img2img work never touches it, because HFDRunPipeline accepts a plain image directly.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Yahweasel/ComfyUI-HF-Diffusers

or search ComfyUI-HF-Diffusers in ComfyUI Manager. Restart afterwards. requirements.txt pins diffusers~=0.36.0.

Gotchas

• It takes the first image only. If you feed a list, it silently grabs element zero. Don't expect batching.
• PIL in, native latent out. Easy to forget which side is which when you're wiring; the input needs the PIL bridge, the output doesn't.
• The VAE must match the model. A VAE from one architecture on another model's latents gives noise, not a bad image. Same rule as every VAE.