Face Detect + Track

Face Detect + Track is the front half of the Temporal Face Detailer pipeline, broken out so you can see what the tracker thinks before any GPU time gets spent on sampling. It takes a video frame batch, detects faces frame by frame, links them into stable identity tracks, and hands you two things: a FACE_TRACKS object and a debug overlay. No diffusion involved - this is pure computer vision, and it runs in a fraction of the time a sampling pass takes.

Why you'd reach for it

The all-in-one Temporal Face Detailer does detection internally, but the split path exists for a reason: tracking runs once, so you can iterate on sampler settings in Tracked Face Detail cheaply and repeatedly. More importantly, you can preview the tracking before committing to an expensive render. The README's advice is blunt and correct: if boxes flicker or drop out in the debug overlay, fix detection (switch detector) before you touch a single denoise knob. Every anti-flicker trick in this pack is downstream of detection being stable.

How it works

Per-frame detection runs through a swappable backend, then detections are linked into tracks with a greedy association on a combined IoU + normalized-centroid-distance cost against a constant-velocity prediction of each track. Tracks tolerate brief occlusion - a face can vanish for up to max_track_gap frames and the bbox and landmarks get linearly interpolated across the gap so the downstream detailer sees a continuous crop sequence.

The detector dropdown is where most of the action is. insightface (RetinaFace, with 5-point landmarks) is the default and best for realistic faces. yunet is OpenCV's small ONNX detector, auto-downloaded to models/tfd. haar is the no-download, no-landmarks fallback. And any ultralytics model you drop in models/ultralytics appears as yolo:... - that's your anime path (e.g. yolo:bbox/face_yolov8m_anime.pt), because realistic-face detectors genuinely miss stylized characters.

The inputs that matter

Most of these you can leave alone, but the handful you'll actually touch:

• detector - the one that decides whether your faces are found at all.
• iou_threshold - minimum overlap to link a detection to a track; raise it if track IDs keep splitting, lower it if faces keep getting missed.
• max_track_gap - how many frames a face may vanish (occlusion) before its track ends, gaps interpolated. Default 10 is generous.
• max_faces - cap on faces per frame (default 4).
• crop_anchor - landmarks (default) centers the crop on the facial-landmark centroid, which jitters far less than the detection box; falls back to bbox when the detector yields no landmarks.
• crop_smoothing - temporal smoothing of the crop window, your anti-jitter dial (0.8 default).
• align_rotation - rotation-registers crops so the eye line is horizontal in every sampled crop; needs a landmark-capable detector, and it's inverse-warped back on paste. Turn it on if heads tilt and you still see wobbly faces.
• detector_device - cpu offloads detection if you need every MB of VRAM.

Outputs

face_tracks (FACE_TRACKS) is the thing you wire onward - into Tracked Face Detail for the sampling pass, or into Face Track Preview to re-draw the overlay later. debug_overlay is an IMAGE showing per-face detection boxes with confidence, the stabilized crop windows, landmarks, track IDs, interpolated gap frames, and the rotation angle - run it through a Preview Image or a Video Combine to eyeball it.

Where people get burned

Anime faces are the classic failure: the realistic detectors either miss them or jitter frame to frame, and the result is flicker that looks like a sampling problem but is actually a detection problem. Install ultralytics, add an anime face model, and watch the overlay go solid. The other gotcha is expecting landmarks from haar - you get none, so crop_anchor falls back to bbox and align_rotation does nothing. If you want those, use insightface or yunet.