Face Performance Lip Refine

Lip sync is where a lot of performance animation quietly dies. You can have perfect jaw rotation and still look like a badly-dubbed movie, because the mouth isn't contacting - teeth show when they shouldn't, the lips float instead of pressing together. MKRFacePerformanceLipRefine is the MKRShift node that takes a base face-motion stream and an aligned audio-feature stream and refines the mouth behavior: jaw openness, lip contact, teeth visibility, even where the eyes should look.

Like the other face-performance nodes, it's a data-processing node, not a renderer. In goes JSON, out comes JSON - specifically a refined frame array your rig or downstream stage consumes. It's the "make it actually lip-sync, not just vaguely move" pass.

How it works

The mechanism lives in lib/face_performance/lip_refine_face.py. It takes base_frames_json (your base face motion - fields like jaw_open, blink_l, blink_r, gaze_yaw, gaze_pitch, optional mouth-pressure values) and audio_frames_json (the same per-frame audio features the other performance nodes use), aligned on frame index. It then refines mouth contact and ROI placement, using the audio to drive when the mouth should be open, how much, and with what emphasis.

The one input that matters most after the two frame arrays is mode, with exactly two choices:

• quality - steadier, more finished output; the mouth tracks with less noise and more deliberate contact. The author's docs note it uses a larger composite feather. This is the one for a final render.
• realtime - more responsive to the audio frame-by-frame, with more jitter. That's the trade - it feels live but it's less clean. Use it for previews or live-driven setups, then switch to quality for the take you keep.

settings_json overrides any LipRefineFaceConfig field; leave it {} unless you've read the config class.

Outputs: refined_frames_json (jaw openness, lip contact, teeth visibility, pupil targets, composite guidance) and summary (a string noting how many frames and which mode ran). Frame count matches the input, so it slots directly between MKRFacePerformanceEyeMotion and the pose/rig stages.

Installing

Part of MKRShift Nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/criskb/MKRShift_Nodes

Restart, or ComfyUI Manager search "MKRShift_Nodes". No extra Python deps - it's pure Python over the standard stack.

What it is not

The author's docs are explicit and it's worth repeating: this works on structured motion data, not raw images. If you feed it pixels expecting a lipsync filter, you'll get an error and a blank stare from the node's validator (it checks that your JSON is a list - base_frames_json and audio_frames_json both must be arrays). It's also not a text-to-speech lipsync tool; it refines an existing motion stream against audio features, it doesn't invent speech from nothing.

And a straight talk on audience: this whole face-performance lane is for people who are already moving structured face data around - riggers, retargeting pipelines, experimental setups. There's no community discussion of it yet and no presets, so if you land here from a Google search expecting plug-and-play mouth sync for your video-diffusion workflow, this is the wrong tool; it's a component for building one.