Pose Keypoint to Mask

Pose Keypoint to Mask turns DWPose face landmarks into a mask of whatever facial region you care about - mouth, lips, chin, eyes, the whole face - so you can use pose detection to drive region-specific work instead of painting masks by hand. It's part of the Mickmumpitz pack's lipsync-for-LTX toolset, and the concrete use case is exactly the annoying one: you've detected a person's face with DWPose, and you want a mask for the mouth so your audio-driven lipsync or VFX only affects that region.

Where a hand-drawn mask is static, this one tracks the detected landmarks - re-run it on the next frame and the mask follows the mouth. That's the whole point of deriving a mask from keypoints rather than drawing it.

How it works

The node reads the POSE_KEYPOINT input, which carries per-frame face landmarks (68-point DWPose face keypoints). For each frame it builds a mask on the frame's canvas, then for your chosen region it takes the landmark indices that define that region's outline and fills the polygon between them - cv2.fillPoly against a blank canvas. The region definitions are fixed landmark sets: mouth uses points 48–59, lips 48–67, chin 5–11 plus 57, eyes 36–47, and so on through 16 named regions including the useful combos like mouth_and_nose, both_eyes, and upper_face.

Then two cleanup knobs: dilate_px expands the mask outward (max-pool), and blur_px feathers the edges (double box-blur) so it fades out instead of ending in a hard line. If person_index points at a person who isn't in the frame - or the frame has no detected face - the node returns an empty mask rather than crashing, which is the right behavior for a per-frame loop where detection can drop out.

The inputs and outputs

• pose_kps (POSE_KEYPOINT, required) - from a DWPose/OpenPose detector. This is a non-standard type, so it has to come from a node that emits it.
• region (default mouth) - which facial region to mask; the full list includes mouth, mouth_inner, lips, chin, lower_face, full_face, nose, mouth_and_nose, right/left/both eyes, right/left/both eyebrows, eyes_and_brows, upper_face.
• dilate_px (default 8) - expand the mask outward by this many pixels.
• blur_px (default 6) - feather the edges with a box blur.
• person_index (default 0) - which detected person to use; 0 = first.

The single output, mask, is a MASK batch with one mask per input frame - wire it into whatever compositing or editing step needs the region.

Installing it

Part of the Mickmumpitz pack - ComfyUI Manager, search "Mickmumpitz", install, restart:

cd ComfyUI/custom_nodes
git clone https://github.com/mickmumpitz/ComfyUI-Mickmumpitz-Nodes.git

It depends on opencv-python (in the pack's requirements) for the polygon fill. No models of its own - the model work is done by whatever DWPose detector feeds it.

Common issues

The usual stumble is the input type: POSE_KEYPOINT isn't an IMAGE, so you can't feed it a pose image (the skeleton render). You need a node that outputs the raw keypoint structure - the pack's own openpose_part_cropper and standard DWPose preprocessing nodes do this. Also watch person_index: in a frame with multiple people, the wrong index quietly gives you an empty mask rather than an error. And remember the 68-point assumption - this node is built for DWPose's face landmark set, so pose data that doesn't carry 68 face keypoints won't produce a region mask.