Arc2Face Encoder Loader

Most SD1.5 pipelines start with a text encoder turning your prompt into a conditioning vector. Arc2Face does something sneakier: it swaps the text encoder for one that accepts a precomputed token embedding, so the model can be conditioned on a face vector instead of words. The Arc2Face Encoder Loader is what brings that custom encoder into your graph. The UNet Loader loads the model; this loads the "mouth" the face gets whispered through.

Here's the mechanism. The encoder is a CLIPTextModelWrapper - a patched transformers CLIPTextModel whose forward can take input_token_embs (precomputed token embeddings) and push them through the normal transformer stack. When a generator runs, it encodes the fixed prompt "photo of a id person", finds the "id" token, and writes the 512-dim ArcFace embedding in at that position (padded up to the encoder's hidden size). The wrapper then produces prompt embeddings where the face vector literally occupies the token slot. That's the whole conditioning trick, and without this loader the generators have no way to speak "face" to the UNet.

Input

One STRING: encoder_path, default "encoder". As with the UNet loader, it's relative - the node looks in models/arc2face_checkpoints/<encoder_path>. The default expects a subfolder named encoder containing the model files. You'll basically never change it.

Output

ARC2FACE_ENCODER - a pack-internal type that wires into both generators alongside the ARC2FACE_UNET from the UNet Loader. The two loaders plus a FACE_EMBEDDING are the entire input stack the generators need.

Install and file placement

Clone the pack and install requirements as usual:

cd ComfyUI/custom_nodes
git clone https://github.com/caleboleary/ComfyUI-Arc2Face.git
pip install -r requirements.txt

The encoder files come from FoivosPar/Arc2Face - encoder/config.json and encoder/pytorch_model.bin - and must live in a nested folder:

ComfyUI/models/arc2face_checkpoints/encoder/
├── config.json
└── pytorch_model.bin

Note it's a pytorch_model.bin (PyTorch weights), not a safetensors - don't go hunting for a .safetensors version of the encoder; the .bin is the real artifact.

Where people get burned

• Flattening the folder structure. The most common failure: someone puts pytorch_model.bin in arc2face_checkpoints/ root next to the UNet, the UNet loader works fine, and only the encoder fails with a from_pretrained error. The loader is looking in the encoder/ subdirectory. Keep the nested tree.
• Don't substitute a stock CLIP encoder. The wrapper's extra arguments are required by the generation path; a normal CLIPTextModel won't accept them and the graph will error or silently produce nonsense.
• Dtype handling is automatic - fp16 on CUDA, fp32 on CPU - so there's nothing to tune there, but it does mean this loader is worth ~a second glance if you're debugging memory: the encoder is small, the UNet is the VRAM hog.

If you're setting up Arc2Face from scratch, do both loaders, feed the extractor a couple of good photos of your subject, and the Generator takes it from there.