HF Diffusers run pipeline
The sampler that speaks HuggingFace (and speaks PIL)
- pipeline
- image
- mask_image
- latents
- prompt_embeds
- pooled_prompt_embeds
- negative_prompt_embeds
- negative_pooled_prompt_embeds
- PIL_IMAGE
- LATENT
If HFDLoadPipeline is the pack's engine, this is the ignition. It takes a loaded HFD_PIPELINE and actually runs it - your prompt goes in, an image (or latent) comes out. In native ComfyUI terms, it's the KSampler of this ecosystem, except it's calling HuggingFace's pipeline() directly instead of driving the sampler primitives.
The required inputs are the ones you'd expect from a sampler:
- prompt - multiline text, defaulting to the astronaut-on-mars classic.
- negative_prompt - optional; if you also wire in negative embeds, the text version is ignored.
- width / height - set to 0 and the pipeline uses the model's default resolution instead of forcing one.
- seed - full 64-bit range.
- num_inference_steps - here's the friendly bit: 0 means "use the pipeline's default step count." So you can leave it at zero and just let the model's own config decide.
- output_type -
pilorlatent. Pick one; only that output is active. That's the author's explicit design: latents or images, never both. - kwargs - JSON for guidance_scale, scheduler options, anything else. This is where your real tuning happens - the example workflows pass things like
{"guidance_scale": 0}here.
The optional inputs are where it gets interesting
- image and mask_image (both
PIL_IMAGE) - plug these in and the same node drives image-to-image and inpainting pipelines. That's the whole img2img path in one node. - latents - start sampling from your own latent (e.g. one you encoded with
HFDVAEEncode). - prompt_embeds / negative_prompt_embeds / pooled_prompt_embeds / negative_pooled_prompt_embeds - the outputs of
HFDEncodePrompt. Wire any in and the node discards the text prompt and uses your pre-encoded tensors.
Outputs and the PIL thing
Two outputs: PIL_IMAGE and LATENT. The LATENT output is native ComfyUI and plugs into HFDVAEDecode or a normal VAEDecode. The PIL_IMAGE output is the gotcha every beginner hits: it's a PIL image, not ComfyUI's IMAGE, so to save it or do anything with it you need a PIL-to-IMAGE bridge node (ComfyUI_Ib_CustomNodes' PILToImage is what every example workflow uses). The author says so in the README and it's the single most common cause of "why won't my SaveImage connect."
Also nice: the code checks for a frames attribute, so video pipelines (returning frames) work through this same node, returning the first frame as the PIL output.
Install
cd ComfyUI/custom_nodes
git clone https://github.com/Yahweasel/ComfyUI-HF-Diffusers
or install ComfyUI-HF-Diffusers via ComfyUI Manager, then restart. requirements.txt pins diffusers~=0.36.0.
Where people get burned
- Missing the PIL bridge. You'll wire PIL_IMAGE into SaveImage and stare at a broken connection. The bridge node is not optional - the README points you to a separate pack for it.
- Both outputs at once? No. You get whichever
output_typeyou chose; the other output is empty. Pick your mode first, then wire. - First run is a download. The model pulls from the HF cache, and a big diffusers repo is multiple gigabytes. It looks hung. It isn't.
Inputs (16)
| Name | Type | Default | Description |
|---|---|---|---|
| pipeline | HFD_PIPELINE | — | |
| prompt | STRING | a photo of an astronaut riding a horse on mars | — |
| negative_prompt | STRING | — | |
| width | INT | 1024 | — |
| height | INT | 1024 | — |
| seed | INT | 10–18446744073709550000 | — |
| num_inference_steps | INT | 00–65536 | — |
| output_type | COMBO | 2 options: pil, latent | |
| kwargs | STRING | — | |
| imageopt | PIL_IMAGE | — | |
| mask_imageopt | PIL_IMAGE | — | |
| latentsopt | LATENT | — | |
| prompt_embedsopt | TENSOR | — | |
| pooled_prompt_embedsopt | TENSOR | — | |
| negative_prompt_embedsopt | TENSOR | — | |
| negative_pooled_prompt_embedsopt | TENSOR | — |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| PIL_IMAGE | PIL_IMAGE | — |
| LATENT | LATENT | — |