Face Track

Face replacement is one of the most-requested AI video jobs, and it keeps tripping people up on the tracking step. You need to know where the face is on every frame, and hand-keyframing a box over a moving head is the worst. Face Track just does it: it detects the face with OpenCV's YuNet detector - which ships inside stock OpenCV, so there's nothing to download by hand - fits a similarity track (translation, rotation, uniform scale) from the stable landmarks, and hands you a per-frame face mask plus the same TRACK_DATA every other node in the pack consumes.

How it works

YuNet finds the face and emits five landmarks: eyes, nose, mouth corners. Only the three most stable ones (eyes + nose) go into the similarity fit, which is exactly the motion a face replacement wants - the head can rotate and scale, but it isn't a plane doing perspective. The tracked region is the convex hull of the landmarks, expanded by expand (default 1.8) to cover the forehead and chin. Frames where detection momentarily drops out hold the nearest solved transform instead of leaving a hole, so you get a continuous track through a blink or a quick turn.

Inputs and outputs

• backend - auto (default), yunet, or mediapipe. YuNet needs nothing extra. Mediapipe is an alternate detector backend, but its wheels don't cover Python 3.14 yet, so most people will stay on auto.
• reference_frame (default 0) - which frame is the reference for the similarity fit.
• expand (default 1.8) - how far the face region extends past the landmark hull. Too tight and you crop the forehead; too loose and you pull in hair and background.
• feather (default 3) - blur on the mask edge, so the replacement blends instead of showing a hard boundary.
• score_threshold (default 0.5) - minimum detection confidence to accept a frame. Lower it for difficult angles, raise it if you're getting false positives.
• hint_x / hint_y / hint_width / hint_height (optional) - a box. In shots with more than one face, setting width/height above 0 picks the face nearest the box's center.

Outputs: track_data (similarity model), preview, and face_mask - a per-frame mask batch ready for compositing or masking an inpaint.

The YuNet model auto-downloads to models/platetrack/ on first run.

The workflow it unlocks

Load Video → Face Track ─┬─> Track Stabilize (crop_roi) ─> [face gen / img2img] ─┐
                          ├──────────────────────────────> Track Destabilize <────┘
                          └── face_mask ──────────────────────────────────────> (feather)

Stabilize locks the face, you regenerate it, Destabilize puts the motion back and blends by the face mask. That's the whole face-replacement pipeline in one paragraph.

Installing it

Same pack - PlateTrack in Manager, or clone + pip install -r requirements.txt. YuNet is bundled in OpenCV, so no extra wheel (the model file itself fetches on first run). Mediapipe only if you explicitly want it: pip install -r requirements-optional.txt.

Where people get burned

• Profiles and extreme angles: YuNet is a frontal-face detector at heart. A head turned 90° may not register, and you'll see the "gaps held from neighbours" message in the console. If the whole shot is a profile, this is the wrong tool.
• The face mask is a rough hull, not a hair matte. Composite at a sensible feather and expect to trim with your own matte for broadcast-grade work.
• Multi-face scenes without a hint: it takes the highest-score face. Use the hint box to be explicit.