PNGTuber Video Upload to Mouth Bundle

If you're here, you probably have a looping character video and want a PNGTuber rig: a mouthless loop, five mouth sprites (closed/half/open/e/u), and a per-frame mouth track that a PNGTuber app can play against your mic. This is the node for that, and it's the one the pack wants you to use first. The pack is called "PromptMaker" because it targets the PromptMaker PNGTuber format, but it writes a general pngtuber.videoMouthBundle.v1 bundle that any player reading that contract can consume. And despite the name, it's fully local - no API calls, no keys, nothing phoning home.

What it does

Drop in a video of your character talking, and this node runs the whole extraction pipeline in one shot: it detects the face and mouth, erases the mouth out of every frame, scores each frame for mouth signal and occlusion, picks the best candidates, and slices them into transparent mouth sprites. Then it writes everything the player needs - loop_mouthless_h264.mp4, mouth_track.json, mouth_sprite_atlas.json, bundle_manifest.json, and the flat sprites plus angle sets from -45° to +45° in 15° steps.

The detection chain is anime_first: it tries the bundled anime-face cascade, refines the mouth region with OpenCV, falls back to MediaPipe FaceMesh, and finally to a YOLO face model if you have one. For a normal anime talking-head loop, you won't think about any of that.

The inputs that matter

• video - the selector. Use the node's upload video button or drag a file onto it. That's the normal path.
• quality_preset - fast_preview caps at 240 frames with stride 2 and 384px sprites; balanced processes the whole clip at 512px; full_quality goes 768px. Start with balanced, switch to fast_preview while you're testing.
• output_dir / asset_id - leave output_dir empty and it writes under ComfyUI's output folder as output/pngtuber_video_mouth/<asset_id>/. asset_id empty means it derives one from the video filename.
• preserve_audio - keep the original audio track on the mouthless loop.
• Optional advanced_video_path lets you type an absolute path for scripts; angle_range_degrees / angle_step_degrees widen or narrow the angle atlas.

Outputs are ten STRING paths - the mouthless_video, the three JSON files, and the five sprite paths. Wire them to a Save node or just read them from the output folder; there's a loadable example at examples/workflows/pngtuber_video_upload_bundle.json.

Install

ComfyUI Manager, searching for ComfyUI-PromptMaker-PNGTuber, or by hand:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/hobi2k/ComfyUI-PromptMaker-PNGTuber.git
cd /path/to/ComfyUI
python -m pip install -r custom_nodes/ComfyUI-PromptMaker-PNGTuber/requirements.txt

Restart ComfyUI. The base requirements are light - mediapipe==0.10.21, opencv-python, Pillow, imageio-ffmpeg, pinned numpy==1.26.4 - and the anime cascade ships in the repo.

One thing the README glosses over, and you'll hit it on the first run: the mouth erase step defaults to the local Qwen/eigen image-edit path, and this node won't silently fall back. The README says "OpenCV TELEA," but the shipped default erase_method is eigen - if the Qwen Image Edit stack (model, text encoder, VAE, and the TextEncodeQwenImageEditPlus node) isn't installed, the run dies with Eigen/Qwen mouth erase failed at frame 0; refusing local CV inpaint fallback. Two ways out: install the Qwen stack (multi-GB downloads, described in the README's generated-mouth section), or use PNGTuber Video Mouth Builder instead - it's the same pipeline but exposes erase_method, so you can set cv2 and run on requirements.txt alone. Both paths produce the same bundle shape.

Where people get burned

The big one: a closed-mouth or low-articulation video. If the clip doesn't show enough mouth-open variation, the node marks the bundle with articulation.requiresModelGeneration: true and writes mouth_generation_inputs/* (reference crop, mask, plan) instead of pretending the forced extraction was good. That's by design - treat the extracted sprites as provisional and run the generated-mouth pass (see PNGTuber Generated Mouth Sprite Applier). Less exotic failures: the mediapipe==0.10.21 pin can collide with a newer MediaPipe another pack installed - install requirements into the same Python env ComfyUI uses, and if a red "mediapipe is required" error pops, that's the cause. And remember to restart after installing so the class registers. If a loaded example shows red LoadImage nodes, that just means placeholder filenames - upload your real files.