MediaPipe Face Mesh (Tracked)

The ControlNet MediaPipe Face Mesh preprocessor that ships with ComfyUI's Auxiliary Preprocessors is fine for a single still. Point it at video with two people in frame and the mesh will blink, jump between faces, and generally act like it forgot who's who. MediaPipeFaceMeshTracked is the drop-in replacement that fixes exactly that: it runs Google's FaceLandmarker and then assigns every detected face a persistent identity, so a face that vanishes for a few frames gets interpolated back into its own slot instead of being mistaken for the other person.

You'd reach for it any time a face-mesh overlay has to be temporally stable - driving a Wan VACE video from a mesh condition, re-animating lip sync, or working with multi-speaker footage. If you've watched a mesh dance between two speakers and wreck the latent video, this is the node that kills it.

How it works

The mechanics are the interesting part. It runs MediaPipe FaceLandmarker in VIDEO mode (Google's tasks API, the Apache-2.0 one the ecosystem reached for when InsightFace's non-commercial licensing got in the way - see the mediapipe panel) to get the 478-point mesh per frame. Then the real work: every detection is matched to a track by centroid distance using Hungarian assignment (scipy.optimize.linear_sum_assignment). Each face gets a stable track ID, and when a track drops out for a few frames, the landmarks are linearly interpolated between the last and next good frame per track - not the whole-frame "everyone blinks" behavior you get from the stock node.

The inputs that matter

The only required input is image. The optional knobs are the usual suspects:

• max_gap (default 8) - how many frames a missing face stays alive for interpolation before the track is retired. This is the one to bump if faces vanish during occlusion.
• match_gate (default 0.6) - max centroid shift between frames as a fraction of face bbox diagonal. Higher = more permissive matching, which is how two faces end up swapping.
• ema_alpha (default 0) - per-track exponential smoothing; 0.4–0.7 for jitter reduction.
• min_face_size (default 64) - drops detections smaller than this in pixels; set 0 to disable.
• max_faces, min_confidence, and draw_iris are self-explanatory.

The single output is an IMAGE - the same rendered mesh overlay the ControlNet node gives you, so it plugs straight into whatever was consuming that. It's genuinely drop-in for that use.

Installing it

ComfyUI Manager (search "lip_mask_from_pose") or:

cd ComfyUI/custom_nodes
git clone https://github.com/Jalen-Brunson/lip_mask_from_pose

then restart ComfyUI. Here's the gotcha the README hides: the pack says "no additional dependencies," and that's true for the pose-based nodes - but this one imports mediapipe and scipy at module load, and there's no requirements.txt for Manager to read. Install them yourself into the python env ComfyUI runs under:

pip install mediapipe scipy

Also, the first run auto-downloads face_landmarker.task (~7 MB) from Google's storage. Watch the path: this node checks /workspace/ComfyUI/models/mediapipe/face_landmarker.task (a container path) or ~/.cache/mediapipe/face_landmarker.task. On a normal home install, make sure it lands in the ~/.cache location, or it'll try to create /workspace and throw. Worth noting the same pack's MouthMeshExtractor downloads the same model to ComfyUI/models/mediapipe/ - the two nodes don't share a path, so you can end up with two copies.

Troubleshooting

• Faces still swapping tracks: push match_gate down toward 0.4, or raise min_confidence a bit.
• Jitter frame to frame: ema_alpha 0.4–0.7 smooths it out, at the cost of a little lag.
• Background faces you don't want: the default min_face_size of 64 already drops small ones; raise it if strays still slip through.