Nodes/ComfyUI-HF-Diffusers/HF Diffusers run pipeline
ComfyUI Node

HF Diffusers run pipeline

The sampler that speaks HuggingFace (and speaks PIL)

By Yahweasel·Created 8 months ago·Updated 7 months ago· 2
HF Diffusers run pipeline
  • pipeline
  • image
  • mask_image
  • latents
  • prompt_embeds
  • pooled_prompt_embeds
  • negative_prompt_embeds
  • negative_pooled_prompt_embeds
  • PIL_IMAGE
  • LATENT
prompta photo of an astronaut riding a horse on mars
negative_prompt
width1024
height1024
seed1
num_inference_steps0
output_type
kwargs

If HFDLoadPipeline is the pack's engine, this is the ignition. It takes a loaded HFD_PIPELINE and actually runs it - your prompt goes in, an image (or latent) comes out. In native ComfyUI terms, it's the KSampler of this ecosystem, except it's calling HuggingFace's pipeline() directly instead of driving the sampler primitives.

The required inputs are the ones you'd expect from a sampler:

  • prompt - multiline text, defaulting to the astronaut-on-mars classic.
  • negative_prompt - optional; if you also wire in negative embeds, the text version is ignored.
  • width / height - set to 0 and the pipeline uses the model's default resolution instead of forcing one.
  • seed - full 64-bit range.
  • num_inference_steps - here's the friendly bit: 0 means "use the pipeline's default step count." So you can leave it at zero and just let the model's own config decide.
  • output_type - pil or latent. Pick one; only that output is active. That's the author's explicit design: latents or images, never both.
  • kwargs - JSON for guidance_scale, scheduler options, anything else. This is where your real tuning happens - the example workflows pass things like {"guidance_scale": 0} here.

The optional inputs are where it gets interesting

  • image and mask_image (both PIL_IMAGE) - plug these in and the same node drives image-to-image and inpainting pipelines. That's the whole img2img path in one node.
  • latents - start sampling from your own latent (e.g. one you encoded with HFDVAEEncode).
  • prompt_embeds / negative_prompt_embeds / pooled_prompt_embeds / negative_pooled_prompt_embeds - the outputs of HFDEncodePrompt. Wire any in and the node discards the text prompt and uses your pre-encoded tensors.

Outputs and the PIL thing

Two outputs: PIL_IMAGE and LATENT. The LATENT output is native ComfyUI and plugs into HFDVAEDecode or a normal VAEDecode. The PIL_IMAGE output is the gotcha every beginner hits: it's a PIL image, not ComfyUI's IMAGE, so to save it or do anything with it you need a PIL-to-IMAGE bridge node (ComfyUI_Ib_CustomNodes' PILToImage is what every example workflow uses). The author says so in the README and it's the single most common cause of "why won't my SaveImage connect."

Also nice: the code checks for a frames attribute, so video pipelines (returning frames) work through this same node, returning the first frame as the PIL output.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/Yahweasel/ComfyUI-HF-Diffusers

or install ComfyUI-HF-Diffusers via ComfyUI Manager, then restart. requirements.txt pins diffusers~=0.36.0.

Where people get burned

  • Missing the PIL bridge. You'll wire PIL_IMAGE into SaveImage and stare at a broken connection. The bridge node is not optional - the README points you to a separate pack for it.
  • Both outputs at once? No. You get whichever output_type you chose; the other output is empty. Pick your mode first, then wire.
  • First run is a download. The model pulls from the HF cache, and a big diffusers repo is multiple gigabytes. It looks hung. It isn't.
Categoryhuggingface-diffusers

Inputs (16)

NameTypeDefaultDescription
pipelineHFD_PIPELINE
promptSTRINGa photo of an astronaut riding a horse on mars
negative_promptSTRING
widthINT1024
heightINT1024
seedINT10–18446744073709550000
num_inference_stepsINT00–65536
output_typeCOMBO2 options: pil, latent
kwargsSTRING
imageoptPIL_IMAGE
mask_imageoptPIL_IMAGE
latentsoptLATENT
prompt_embedsoptTENSOR
pooled_prompt_embedsoptTENSOR
negative_prompt_embedsoptTENSOR
negative_pooled_prompt_embedsoptTENSOR

Outputs (2)

NameTypeDescription
PIL_IMAGEPIL_IMAGE
LATENTLATENT