PNGTuber Frame Sequence Mouth Builder

Sometimes your character isn't a video file at all - it's a folder of transparent frames, straight out of a render (THA3 idle loops, Live2D exports, image-sequence renders). PNGTuberFrameSequenceMouthBuilder is the node for that case. It reads a directory of f*.png frames, tracks the mouth across them, picks the best frames for each mouth shape, and writes a per-frame mouth track plus the five sprites. No video decoding, no frame interpolation, no re-encoding - the frames are the source of truth.

One thing to internalize: this node never creates mouthless frames. It produces the track and the mouth sprites, full stop. The pack's intended pairing is a mouthless THA3 render (generated from an eigen-erased source image, which is what PNGTuber Quality Asset Builder produces) with this node supplying the motion. There's also a thoughtful extra: alongside the normal sprites it writes mouth_cover (skin-only oval patches that intentionally avoid copying jaw/face line art) and mouth_tts (cover + sprite composites), so you can play mouth shapes over footage that still contains the original mouth without doubling up.

How tracking works here

Detection defaults to manual_motion_refine: you give it one manual_base_mouth_bbox (x0,y0,x1,y1) on the first frame, and it propagates that quad across frames by refining against local motion and mouth-signal cues. That's more robust than re-detecting faces every frame on a stylized render. face_yolo_refine re-detects per frame with a YOLO face model when available; legacy_fallback is the older cascade path.

Inputs and outputs

• frames_dir - the folder of f*.png frames (the glob is specific, so name your frames f0001.png-style).
• fps (24) - stamped into the track for the player.
• sprite_size (512), track_quad_scale (1.8) - sprite resolution and quad size.
• manual_base_mouth_bbox - highly recommended for manual_motion_refine.
• detection_mode - manual_motion_refine (default), face_yolo_refine, legacy_fallback.

Outputs: mouth_track_json, bundle_manifest_json, and the five sprite paths (mouth_closed, mouth_half, mouth_open, mouth_e, mouth_u). The manifest is pngtuber.frameSequenceMouthBundle.v1 and records which frames fed which shape.

Install

Same pack, same routine:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/hobi2k/ComfyUI-PromptMaker-PNGTuber.git
cd /path/to/ComfyUI
python -m pip install -r custom_nodes/ComfyUI-PromptMaker-PNGTuber/requirements.txt

Restart after. This node builds sprites straight from your frames, so it doesn't need the Qwen/eigen stack to produce output - a nice relief if you've been dodging the model downloads. It needs OpenCV + MediaPipe from requirements.txt, which is it.

Common issues

• "No frames found" - the folder doesn't match the f*.png pattern. Rename your frames or point at the right directory.
• "No mouth detected at frame N" - motion refinement lost the mouth (turn, occlusion, or a bad base bbox). Fix manual_base_mouth_bbox first; that's the anchor everything propagates from.
• Jittery mouth on turns - the quad follows motion, not a re-detected face, so large head rotations will drift. face_yolo_refine handles those better if you have the model; otherwise trim your loop to the range where the character faces the camera.