Lip Mask from Pose Keypoints

The flagship node this pack is named after - and honestly the one that earned the name. LipMaskFromPose takes the POSE_KEYPOINT output of a DWPose preprocessor and turns it into a precise mouth/lip mask you feed straight into video inpainting. Fix a smile, swap a tooth, change lipstick across a clip, re-dub a take where the mouth moves wrong: mask the mouth, let Wan 2.2 VACE (or a KSampler) regenerate only that region, leave everything else untouched.

That's the whole point of it in the wider workflow. VACE needs a mask to know where to edit, and drawing a mouth mask by hand across 120 frames is misery. This node does it from the keypoints you're already computing for pose control, at every frame, automatically.

How it works

DWPose returns face landmarks in the iBUG 300-W 68-point convention, and the lip region is a clean subset of it: the outer lip contour is landmarks 48–59 (12 points) and the inner mouth opening is 60–67 (8 points). The node builds polygons from those, fills them on a blank canvas at the video's resolution, and - here's the part people miss - the mask_mode dropdown decides which polygon math you get:

• inner_mouth - the mouth cavity only (teeth/tongue/dark interior). Ideal for teeth work without touching lip shape.
• outer_lips - the full lip region including lip skin.
• lips_only - outer minus inner: just the lip flesh, for re-coloring lips while preserving the interior.
• outer_expanded, top_lip_only, bottom_lip_only - the more surgical variants.

Beyond that, it's defensive by design. When someone puckers, kisses, or sticks a lollipop in front of the camera, the inner mouth polygon collapses to near nothing and a normal mask would implode. With auto_fallback on (default), the node detects that collapse via fallback_area_threshold - inner area as a ratio of outer area, default 0.15 - and swaps in an ellipse built from stable lip anchors (mouth corners, cupid's bow, lower lip bottom), padded by fallback_pad_h/fallback_pad_v. Same trick repeats across the pack's nodes, and it's the difference between a mask that survives a weird frame and one that doesn't.

The inputs and outputs that matter

Required: pose_keypoint - wire it from DWPreprocessor's POSE_KEYPOINT output with detect_face enabled. mask_mode as above. Then the handful you'll actually touch:

• blur_radius (default 5) - Gaussian feathering on mask edges; 0 = hard edge. Feather before video inpainting or you'll see the seam.
• grow_pixels (default 0) - dilate the mask outward; a couple of pixels stops the inpaint from hugging the lip boundary too tight.
• person_index (default 0) - which detected person; -1 unions all people, which is the setting you want for a group shot.
• confidence_threshold (default 0.3) - skip frames where average lip keypoint confidence drops below this.

Outputs: mask (the one you wire into VACE Inpaint / KSampler), plus three lists: centroid_x_list/centroid_y_list (per-frame lip centroids - the README's suggestion is feeding these as SAM-2 point prompts to refine the mask) and fallback_active_list (which frames actually used the fallback ellipse, so you can see when the mask is really an ellipse).

Installing it

Manager (search "lip_mask_from_pose") or:

cd ComfyUI/custom_nodes
git clone https://github.com/Jalen-Brunson/lip_mask_from_pose

restart, done. This node is pure numpy/cv2/torch - genuinely zero extra dependencies. The dependency you do need for its input is ComfyUI ControlNet Auxiliary Preprocessors for DWPreprocessor; that pack is basically required for any serious ControlNet workflow anyway.

Troubleshooting

• Blank mask out of nowhere: you connected POSE_KEYPOINT but forgot detect_face=enable on the DWPreprocessor - the face landmarks aren't in the payload, so there's nothing to build.
• Mask looks like a blob on certain frames: that's fallback_active_list doing its job; widen fallback_pad_h/fallback_pad_v if the ellipse undershoots the lips.
• Wrong person masked: set person_index correctly, or use -1 for everyone.
• Edgy, visible seams: raise blur_radius and grow_pixels a little - hard masks are the classic VACE seam cause.