PNGTuber Auto Eigen Mouth Builder

Here's a subtle failure mode that only shows up once you've actually built a PNGTuber rig: when you cut mouth sprites from different frames of a video, each mouth is a different moment of the same character - slightly different lighting, expression, head angle. Play them in sequence and the mouth shimmers between "talking" and "someone else's mouth." PNGTuberAutoEigenMouthBuilder is the answer to that: it generates all five mouth shapes from one reference with the Qwen/eigen edit stack, so the sprite identity stays locked, and then uses the video (or frame sequence) only to supply the per-frame mouth track. The "Auto" is real too - it sniffs source_mode from the input: an image extension, a video extension, or a directory of frames.

Image, video, or frames

• Image input: localize the mouth, erase it, generate five full-source eigen/Qwen edits, extract mouth-only sprites, write a static track.
• Video / frame-sequence input: the first frame feeds the eigen/Qwen sprite generation, then the mouth bbox is tracked across the remaining frames for TTS placement. Same bundle shape either way - pngtuber mouth bundle with a mouth_track.json.

This is effectively the image-mouth builder and the frame-sequence builder merged under one "do it right" policy, which is why it defaults to a bigger sprite_size (768) than the other builders. It's the node to pick when the mouth is the center of attention on stream.

Inputs and outputs

• source_path - image, video, or folder; source_mode (auto/image/video/frames) lets you force it.
• mouth_localization - eigen_qwen_then_yolo (default), eigen_qwen, eigen_qwen_only, face_yolo_refine, legacy_fallback. Same menu as the image mouth builder.
• erase_scale (1.65) and qwen_mask_scale (0.72) - erase size and the Qwen edit-mask size. If generated mouths bleed past the mouth area, lower qwen_mask_scale.
• frame_stride (1) and max_frames (0 = all) - how many frames get tracked; the sprites are already fixed by the first frame, so you can afford to stride for speed.
• seed (42), manual_mouth_bbox (x0,y0,x1,y1) - the usual generation seed and the art-direction override.

Outputs: bundle_manifest_json, mouth_track_json, the five sprite paths (mouth_closed/mouth_half/mouth_open/mouth_e/mouth_u), and mouthless_asset (the erased source).

Install and the honest part

Same pack install - Manager search for ComfyUI-PromptMaker-PNGTuber, or clone + pip install -r requirements.txt + restart. Then the catch: everything that makes this node good - the erasing and the sprite generation - is the local Qwen/eigen stack, so a fresh install without those models will fail the moment it tries to load them. There's no cv2 fallback on the generation side, because a cv2 blob would defeat the entire point. Budget the model downloads up front (Qwen-Image-Edit model, Qwen text encoder + VAE, TextEncodeQwenImageEditPlus node), and once it runs you get the one consistent mouth set that the frame-slicing builders can't guarantee.

Troubleshooting

• "Cannot infer source_mode" - the extension wasn't recognized; set source_mode explicitly.
• "Eigen/Qwen ... failed" - stack missing or filenames don't match. The code loads specific assets (qwen-image-edit-2511 GGUF, the Lightning 4-step LoRA, the eigen-banana LoRA), so a generic Qwen install may not satisfy it.
• Mouth drifting across the loop - that's the track, not the sprites; the mouth bbox propagation across frames will drift on fast turns. Nudge track_quad_scale or consider manual_mouth_bbox for a cleaner anchor.