Mouth Shape Compositor (Tracked)

This is the node that makes the whole TrentNodes lip-sync pipeline feel like it was worth building. Given three things - your video frames, your nine mouth-shape images, and a per-frame list of which shape to show - it draws the mouths on. The "Tracked" in the name is the part that matters: it doesn't slap the mouth at a fixed spot, it follows the character, so you can lip-sync a head that turns, bobs, or moves across the frame.

How it works

The compositor needs to know where the mouth goes on each frame, and it gives you two ways to provide that, selected by tracking_mode:

• points - wire a point_sequence (per-frame x,y coordinates from this pack's Point Tracker, or any source that speaks the same POINT_SEQUENCE type). The mouth is centered on that point.
• masks - wire per-frame tracking_masks. The node finds each mask's centroid and centers the mouth there. This is the mode the README suggests for SAM3-derived mouth-region masks, and it's the more robust one if your character's face changes shape, because the mask tracks the actual mouth region instead of one pixel.

tracking_mode defaults to auto, which just means "use whichever of the two is connected," with points winning if both are. Leave it on auto and forget about it.

Before compositing, each mouth shape image gets its background removed so you're pasting the lips, not a box. remove_background is on by default, and bg_method picks BiRefNet (accurate, but downloads weights and runs a segmentation pass on first use) or color_key (instant, but only works when the shapes sit on a solid white or black background). The code falls back from BiRefNet to color keying if the segmentation returns nothing, so a failure isn't a hard crash.

The inputs that matter

• mouth_shapes - a batch of exactly 9 images in A-H, X order. This comes straight out of the Mouth Shape Loader.
• mouth_sequence - per-frame indices into those 9, which is what Phoneme To Mouth Shapes produces.
• scale and offset_x / offset_y - your tune-up knobs. A shape that's too big or sitting a few pixels off the mouth line is normal on the first pass; fix it here rather than re-exporting art.

The three outputs are frames (your video with mouths composited), mouth_rgba (the mouths on transparent, for your own compositing pass), and mouths_cropped (the nine shapes, background removed and cropped to their bounding boxes - handy for previewing what you actually loaded). output_mouth_rgba and output_cropped_mouths gate the last two; leave them on only when you need them, since BiRefNet already did the expensive part.

Installing it

All of these nodes ship in one pack, so install TrentNodes once:

cd ComfyUI/custom_nodes
git clone https://github.com/TrentHunter82/TrentNodes.git
cd TrentNodes
pip install -r requirements.txt

ComfyUI Manager works too - search "Trent Nodes". One real gotcha: the author renamed the repo on day one, which left two registry entries and made Manager flag the pack as "unsafe" for some people. If Manager refuses, the manual clone above always works. The requirements pull in OpenCV, numpy, pillow, matplotlib, and vosk; transformers (for BiRefNet) is in there too, and its weights download on first use.

Where people get burned

• Wrong shape count. The node raises if it gets fewer than 9 mouth shapes, so a half-loaded folder fails loudly, not subtly.
• tracking_mode says points but nothing's wired. It throws a clear "no point_sequence provided" error. If you want a mode that tolerates empty inputs, that's what auto is for.
• Shapes following but misplaced. That's scale and the offsets, every time. Tune them before you blame the tracker.