ComfyUI-Sapiens2

ComfyUI custom nodes for Meta Sapiens2 human-centric vision models.

ComfyUI-Sapiens2 brings the Sapiens2 downstream models into native ComfyUI workflows: human parsing / segmentation, surface normals, point maps, and 308-keypoint pose estimation. The project is intentionally self-contained. It vendors the standalone Sapiens2 backbone and implements task heads, preprocessing, checkpoint discovery, model loading, visualization, mask generation, and ComfyUI node registration inside this repository.

Workflow Preview

<img width="1169" height="738" alt="ComfyUI-Sapiens2 workflow preview" src="https://github.com/user-attachments/assets/2766821d-73a1-4dbf-839d-53d5d456616f" />

An importable example workflow is included at examples/Sapiens2.json. Drag it into ComfyUI, replace the placeholder input image with your own image, and run the graph.

Highlights

• Native ComfyUI nodes; no upstream Sapiens2 repository is required at runtime.
• Automatic checkpoint discovery under ComfyUI/models/sapiens2.
• Supports seg, normal, pointmap, pose, and pretrain tasks.
• Supports 0.4B, 0.8B, 1B, and optional 5B downstream checkpoints.
• Segmentation mask class is selected by name from a dropdown.
• Works with single images and ComfyUI IMAGE batches, including video frames decoded by video loader nodes.
• Includes download helpers for Hugging Face and hfd.sh.

Nodes

Sapiens2 Model Loader

Loads a Sapiens2 model and returns a reusable SAPIENS2_MODEL object.

| Input | Description | | --- | --- | | task | seg, normal, pointmap, pose, or pretrain. | | arch | sapiens2_0.4b, sapiens2_0.8b, sapiens2_1b, sapiens2_5b; pretrain also supports sapiens2_0.1b. | | checkpoint | Dropdown populated from discovered model files. auto matches by task and architecture. | | checkpoint_path | Optional manual path. Overrides the dropdown when non-empty. | | precision | auto, bf16, fp16, or fp32. | | device | auto, cuda, or cpu. |

Sapiens2 Inference

Runs inference on a ComfyUI IMAGE tensor and returns visualization, mask, and metadata.

| Output | Description | | --- | --- | | IMAGE | Visualization image. For segmentation, this is the colored parsing overlay. | | MASK | Binary mask for the selected segmentation class, pose keypoints, valid point map, or full normal map. | | STRING | Metadata JSON with task, checkpoint, per-image classes / scores / statistics. |

For seg, use segmentation_mask_class:

• -1: All foreground returns all non-background classes as one mask.
• A specific class such as 4: Hair or 23: Upper_Clothing returns only that semantic part.

Task Support

| Task | What It Produces | Practical Use | | --- | --- | --- | | seg | 29-class human parsing map plus selected binary mask. | Clothing masks, hair masks, face / hand isolation, full person masks. | | pose | 308-keypoint pose visualization plus keypoint mask and JSON. | Pose visualization, body / face / hand landmark metadata. | | normal | Surface normal RGB visualization. | Geometry-aware conditioning, inspection, relighting pipelines. | | pointmap | Point/depth-style visualization and valid-point mask. | Human-centric geometry and depth-style workflows. | | pretrain | First feature channels visualized for inspection. | Research and debugging. Not required for normal downstream use. |

Dense tasks use Sapiens2's 1024x768 preprocessing resolution and ImageNet normalization. The pose node uses the 308-keypoint heatmap head and UDP-style decoding. It treats the full input image as the person crop, so it does not require the DETR person detector used by the upstream demo scripts.

Requirements

• ComfyUI installed and working.
• Python 3.10 or newer.
• PyTorch installed through your ComfyUI environment.
• CUDA GPU strongly recommended. CPU works only for small tests and is slow.
• Enough VRAM for the selected architecture. Start with sapiens2_0.4b.

The custom node dependencies are intentionally small:

numpy
Pillow
safetensors
huggingface_hub

PyTorch is not pinned here because ComfyUI already manages the correct Torch build for your CUDA / platform.

Installation

1. Clone into ComfyUI custom nodes

From your ComfyUI root:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/starsFriday/ComfyUI-Sapiens2.git
cd /path/to/ComfyUI

2. Install Python dependencies

Use the same Python environment that runs ComfyUI.

Linux / venv example:

cd /path/to/ComfyUI
python -m pip install -r custom_nodes/ComfyUI-Sapiens2/requirements.txt

ComfyUI portable Windows example:

cd ComfyUI
python_embeded\python.exe -m pip install -r custom_nodes\ComfyUI-Sapiens2\requirements.txt

3. Create model folders

The install hook creates these folders automatically when ComfyUI loads the node. You can also create them manually:

mkdir -p models/sapiens2/{seg,normal,pointmap,pose,pretrain}

4. Restart ComfyUI

After restart, search for:

Sapiens2 Model Loader
Sapiens2 Inference

Model Download

Official Sapiens2 checkpoints are hosted on Hugging Face. This node expects weights under:

ComfyUI/models/sapiens2/

Recommended layout:

ComfyUI/models/sapiens2/
  seg/sapiens2_0.4b_seg.safetensors
  normal/sapiens2_0.4b_normal.safetensors
  pointmap/sapiens2_0.4b_pointmap.safetensors
  pose/sapiens2_0.4b_pose.safetensors

The model loader also scans custom_nodes/ComfyUI-Sapiens2/models, but the shared ComfyUI/models/sapiens2 location is recommended.

Recommended Download

This downloads the practical downstream inference set:

• tasks: seg, normal, pointmap, pose
• architectures: 0.4b, 0.8b, 1b
• excludes: optional 5b and pretrain

cd /path/to/ComfyUI
python custom_nodes/ComfyUI-Sapiens2/scripts/download_models.py

Download only the smallest useful set:

python custom_nodes/ComfyUI-Sapiens2/scripts/download_models.py --arches 0.4b

Download only one task:

python custom_nodes/ComfyUI-Sapiens2/scripts/download_models.py --tasks seg --arches 0.4b 0.8b

hfd.sh Download

If normal Hugging Face downloads are slow or unstable, use hfd.sh:

cd /path/to/ComfyUI
bash custom_nodes/ComfyUI-Sapiens2/scripts/download_models_hfd.sh

If hfd.sh is not on PATH, set it explicitly:

HFD=/path/to/hfd.sh bash custom_nodes/ComfyUI-Sapiens2/scripts/download_models_hfd.sh

To use a Hugging Face mirror:

HF_ENDPOINT=https://hf-mirror.com HFD=/path/to/hfd.sh bash custom_nodes/ComfyUI-Sapiens2/scripts/download_models_hfd.sh

Optional 5B / Pretrain Weights

The 5B and pretrain weights are not required for normal downstream inference. They are large and should be downloaded only when you explicitly need them.

With download_models.py:

python custom_nodes/ComfyUI-Sapiens2/scripts/download_models.py \
  --tasks pretrain seg normal pointmap pose \
  --arches 0.1b 0.4b 0.8b 1b 5b

With hfd.sh:

INCLUDE_OPTIONAL=1 HFD=/path/to/hfd.sh bash custom_nodes/ComfyUI-Sapiens2/scripts/download_models_hfd.sh

The copied upstream model list is available in MODEL_ZOO.SAPIENS2.md.

Basic Workflow

An example workflow is included:

custom_nodes/ComfyUI-Sapiens2/examples/Sapiens2.json

You can drag this file directly onto the ComfyUI canvas. It demonstrates a segmentation setup with one model loader feeding two inference nodes, each selecting a different semantic class mask.

Image

Load Image
  -> Sapiens2 Model Loader
  -> Sapiens2 Inference
  -> Preview Image / Save Image / mask consumer node

For segmentation:

1. Set task to seg in Sapiens2 Model Loader.
2. Pick an installed architecture, usually sapiens2_0.4b.
3. Leave checkpoint as auto, or select a specific checkpoint.
4. In Sapiens2 Inference, choose segmentation_mask_class.
5. Use the MASK output in downstream inpainting, compositing, or masking nodes.

Video

This node does not accept a native video file object. It accepts ComfyUI IMAGE tensors. Most ComfyUI video loader nodes decode video into an IMAGE batch, and that batch can be connected directly.

Video Loader
  -> IMAGE batch
  -> Sapiens2 Inference
  -> IMAGE batch / MASK batch
  -> Video Combine

FPS, audio, and container metadata should be handled by your video loader and video combine nodes. Sapiens2 processes the frames only.

Segmentation Classes

The segmentation checkpoint predicts 29 classes:

| ID | Class | ID | Class | ID | Class | | --- | --- | --- | --- | --- | --- | | 0 | Background | 1 | Apparel | 2 | Eyeglass | | 3 | Face_Neck | 4 | Hair | 5 | Left_Foot | | 6 | Left_Hand | 7 | Left_Lower_Arm | 8 | Left_Lower_Leg | | 9 | Left_Shoe | 10 | Left_Sock | 11 | Left_Upper_Arm | | 12 | Left_Upper_Leg | 13 | Lower_Clothing | 14 | Right_Foot | | 15 | Right_Hand | 16 | Right_Lower_Arm | 17 | Right_Lower_Leg | | 18 | Right_Shoe | 19 | Right_Sock | 20 | Right_Upper_Arm | | 21 | Right_Upper_Leg | 22 | Torso | 23 | Upper_Clothing | | 24 | Lower_Lip | 25 | Upper_Lip | 26 | Lower_Teeth | | 27 | Upper_Teeth | 28 | Tongue | | |

Select -1: All foreground to merge every non-background class into one mask.

Checkpoint Resolution

When checkpoint is auto, the loader searches for a file matching the selected task and architecture. For example:

task = seg
arch = sapiens2_0.4b

will match:

models/sapiens2/seg/sapiens2_0.4b_seg.safetensors

Manual override is always available through checkpoint_path. Absolute paths and paths relative to the ComfyUI root are supported.

Advanced users can also point the loader at a shared external directory with:

export SAPIENS_CHECKPOINT_ROOT=/path/to/sapiens2/checkpoints

Troubleshooting

The node does not appear in ComfyUI

Restart ComfyUI and check the console log. Then confirm the folder is exactly:

ComfyUI/custom_nodes/ComfyUI-Sapiens2

Install dependencies in the same Python environment used by ComfyUI:

python -m pip install -r custom_nodes/ComfyUI-Sapiens2/requirements.txt

checkpoint not found

Confirm the model file is under the expected task directory:

ComfyUI/models/sapiens2/seg/
ComfyUI/models/sapiens2/normal/
ComfyUI/models/sapiens2/pointmap/
ComfyUI/models/sapiens2/pose/

If needed, paste the full file path into checkpoint_path.

CUDA out of memory

Use a smaller architecture first:

sapiens2_0.4b

Then reduce batch size or process fewer video frames at a time. The 5B models are optional and require substantially more VRAM.

Downloads are slow or interrupted

Use the hfd.sh workflow or rerun download_models.py; existing complete files are skipped by the download scripts.

Project Layout

ComfyUI-Sapiens2/
  nodes.py                         ComfyUI node definitions
  install.py                       Creates model directories on load
  requirements.txt                 Runtime Python dependencies
  examples/Sapiens2.json           Importable example ComfyUI workflow
  sapiens2/
    modeling.py                    Model heads, loading, preprocessing, postprocessing
    standalone/sapiens2.py         Vendored standalone Sapiens2 backbone
    assets/keypoints308.py         Pose metadata
  scripts/
    download_models.py             Hugging Face downloader
    download_models_hfd.sh         hfd.sh downloader
  MODEL_ZOO.SAPIENS2.md            Copied upstream model list
  LICENSE.SAPIENS2.md              Upstream Sapiens2 license

Notes

• This project focuses on inference inside ComfyUI workflows.
• It does not train or fine-tune Sapiens2 models.
• It does not bundle model weights.
• The vendored upstream Sapiens2 files remain under the upstream license in LICENSE.SAPIENS2.md.

Acknowledgements

This project is built around Meta's Sapiens2 models and architecture. Please refer to the upstream Sapiens2 project and model cards for model details, intended use, and licensing requirements.

License

The custom node integration code is provided under the license of this repository. Vendored Sapiens2 source files and model references are governed by the upstream Sapiens2 license included in LICENSE.SAPIENS2.md.