ComfyUI Extension: ComfyUI_PuLID_Flux_ll

Authored by lldacing

Created 10 months ago

Updated 7 months ago

429 stars

The implementation for PuLID-Flux, support TeaCache, no model pollution.

Custom Nodes (8)

README

中文文档

Solved ComfyUI-PuLID-Flux model pollution problem.
Supported use with TeaCache (Need use with ComfyUI_Patches_ll).
Supported use with Comfy-WaveSpeed, supported by Comfy-WaveSpeed in commit-36ba3c8.
Supported simple use with First Block Cache (Can use with ComfyUI_Patches_ll).

Must uninstall or disable ComfyUI-PuLID-Flux and other PuLID-Flux nodes before install this plugin. Due to certain reasons, I used the same node's name ApplyPulidFlux.

Need upgrade ComfyUI Version>=0.3.7

Update logs

2025.02.19

Fix: when selecting a face from multiple faces as a reference, embeddings and alignment features maybe not from the same face.

2025.02.18

Supported selecting a face from multiple faces as a reference. Example workflow.

2025.01.27

Changed the model path of facexlib to ComfyUI/models/facexlib/.
When automatically downloading, modify the path of Antelope v2 model to ComfyUI/models/insightface/models/antelopev2/.
Changed the model path of EVA_CLIP_L_14_336 to ComfyUI/models/clip/.

Preview (Image with WorkFlow)

save api extended

Install

Manual

    cd custom_nodes
    git clone https://github.com/lldacing/ComfyUI_PuLID_Flux_ll.git
    cd ComfyUI_PuLID_Flux_ll
    pip install -r requirements.txt
    # restart ComfyUI

Tips:

If you use ComfyUI_windows_portable and encounter the following error, please see https://github.com/deepinsight/insightface/issues/2576

insightface/thirdparty/face3d/mesh/cython/mesh_core_cython.cpp(36): fatal error C1083: 无法打开包括文件: “Python.h”: No such file or directory
      error: command 'd:\\installed\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.42.34433\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for insightface
Failed to build insightface

Models

Available Flux models

32bit/16bit (~22GB VRAM): model, encoder
8bit gguf (~12GB VRAM): model, encoder
8 bit FP8 e5m2 (~12GB VRAM): model, encoder
8 bit FP8 e4m3fn (~12GB VRAM): model, encoder
Clip and VAE (for all models): clip, vae

For GGUF models you will need to install ComfyUI-GGUF

PuLID models

Download PuLID-Flux => ComfyUI/models/pulid/.
(Support auto-download) Download EVA02-CLIP-L-14-336 => ComfyUI/models/clip/.
(Support auto-download) Download all models like *.onnx from AntelopeV2 => ComfyUI/models/insightface/models/antelopev2/.
(Support auto-download) Download parsing_bisenet, parsing_parsenet and Resnet50 => ComfyUI/models/facexlib/.

Nodes

PulidFluxModelLoader
- See chapter PuLID models
PulidFluxInsightFaceLoader
- See chapter PuLID models
PulidFluxEvaClipLoader
- See chapter PuLID models
ApplyPulidFlux
- Solved the model pollution problem of the original plugin ComfyUI-PuLID-Flux
- attn_mask ~~may not work correctly (I have no idea how to apply it, I have tried multiple methods and the results have been not satisfactory)~~ works now.
- If you want use with TeaCache, must put it before node FluxForwardOverrider and ApplyTeaCachePatch.
- If you want use with Comfy-WaveSpeed, must put it before node ApplyFBCacheOnModel.
FixPulidFluxPatch (Deprecated)
- If you want use with TeaCache, must link it after node ApplyPulidFlux, and link node FluxForwardOverrider and ApplyTeaCachePatch after it.
PulidFluxOptions
- input_faces_order - Sorting rule for detected bboxes.
  - left-right: Sort the left boundary of bbox by column from left to right.
  - right-left: Reverse order of left-right (Sort the left boundary of bbox by column from right to left).
  - top-bottom: Sort the top boundary of bbox by row from top to bottom.
  - bottom-top: Reverse order of top-bottom (Sort the top boundary of bbox by row from bottom to top).
  - small-large: Sort the area of bbox from small to large.
  - large-small: Sort the area of bbox from large to small.
- input_faces_index - The target index of the sorted bboxes.
- input_faces_align_mode - Choose the detection method for aligning facial features.
  - 0: Old version method, When there is a face in an image, the selected facial embedding amount and alignment features maybe not consistent.
  - 1: Keep the selected facial embedding amount and alignment features consistent.
  - There is a slight difference between the two mode, with the align_face value of 1 resulting smaller area than the embed_face value of 0.
PulidFluxFaceDetector
- Can check the facial features applied in ApplyPulidFlux.
- When input_faces_align_mode = 0, the embed_face and align_face should be the same face, but they are generated by different detectors, and the number detected may be not consistent, so they may be not the same face.
- When input_faces_align_mode = 1, the embed_face and align_face are always the same face, they are generated by same detectors.
- face_bbox_image - Draw the detected facial bounding box (the result of the embed_face's detector).

Thanks