MiniMatesNode

You hand it a photo of a person and an audio clip, and it hands back a video of that person's face speaking your audio. That's the whole job, and MiniMatesNode does it in a single node - no ControlNet, no diffusion sampling loop to babysit, just image in, audio in, mp4 out.

It's one of a line of "digital human" wrappers from AIFSH, the same Chinese-ecosystem author behind F5-TTS, GLM-4-Voice, hallo2 and SeedV nodes. MiniMates itself (from kleinlee) is a lightweight talking-head model aimed at the virtual-presenter crowd - think news-reader lip-sync, not uncanny deepfake cinema. Be ready for an obscure corner of the ecosystem: the README is Chinese-first, the tutorials live on bilibili, and the weights ship from a Quark cloud drive instead of HuggingFace. If you've already fought an AIFSH pack, you know the drill.

How it actually works

The node is a thin shell. It writes your avatar to a temp PNG, saves the audio to a WAV, then shells out to a Python script that does the real work:

• MediaPipe detects a 478-point face mesh on your photo.
• An LSTM audio model (lstm_model_epoch_590.pth) turns the waveform into mouth/expression motion.
• FreeFace (FreeFace/epoch_40.pth) re-renders the face with the new mouth, and the result is texture-mapped onto a 3D head and drawn through a hidden OpenGL/GLFW window.
• Frames get written to an mp4 (mp4v, 25 fps) at your avatar's resolution, dropped into ComfyUI/output.

One thing to know: the whole render runs in a subprocess via os.system, and the node returns the output path whether or not that subprocess actually succeeded. That's the number-one source of "it ran but I got a broken video," more on that below.

The inputs that matter

• avator (IMAGE) - your still photo. Yes, it's misspelled "avator" in the schema; that's literally the field name on the node. Front-facing, well-lit faces work best, naturally.
• driving_audio (AUDIO) - feed it from ComfyUI's core Load Audio node, or from any AIFSH TTS node that emits the AUDIO type (F5-TTS, GLM-4-Voice, …). That's the classic pipeline: TTS straight into a talking head.
• if_matting (BOOLEAN, default false) - the author's tooltip: "if matting the person from avator image." When true it cuts the person out of the background with a MODNet model (modnet.onnx) before animating, so only the subject moves. This is the one input where you can pick wrong: leave it off and the background rides along with the motion.
• driving_video (VIDEO, optional) - pass a clip of a head moving and the node extracts its motion into a cached .template file, then animates your avatar with those head movements. Leave it empty and you get a mostly-still head with a moving mouth.
• Output: VIDEO - a path to the mp4. Wire it into a video preview/save node (VideoHelperSuite's VHS_VideoOutput or core Save Video) to see it in the UI; otherwise check ComfyUI/output for the file.

Install

ComfyUI Manager (search "MiniMates-ComfyUI" under AIFSH_MiniMates) or:

cd ComfyUI/custom_nodes
git clone https://github.com/AIFSH/MiniMates-ComfyUI

Then restart. The heavy lift is the dependencies - requirements.txt pulls in mediapipe, onnxruntime, librosa, kaldi_native_fbank, the OpenGL pair glfw + PyOpenGL, and scikit-learn. The real friction, though, is the weights: they live behind a Chinese Quark drive link in the README (no HF mirror shipped), and they have to land in ComfyUI/models/AIFSH/ so the code finds MiniMates/modnet.onnx, MiniMates/FreeFace/epoch_40.pth, MiniMates/lstm/lstm_model_epoch_590.pth and MiniMates/pca_mediapipe.pkl. Get the folder layout wrong and you'll hit file-not-found errors inside a subprocess - which, remember, the node won't surface.

Where people get burned

• Silent subprocess failures. The node never checks the exit code of os.system. If you get a zero-byte or corrupt mp4, read the ComfyUI console - it prints the exact command it ran - then run that command by hand to see the real traceback.
• No display, no render. The head is drawn through a GLFW window (created invisible via glfw.VISIBLE = FALSE, but it still needs a working GL context). On a headless server without X/Wayland/EGL this dies at glfw.init(). This is a desktop-with-GPU node, not a cloud-box node.
• if_matting = true but no modnet.onnx → crash on matting load.
• Stale project. Last commit was late 2024 and the docs assume you read Chinese. It still works if you're patient, but don't expect English support threads.

Verdict: for a quick "make this person say this audio" job it's genuinely nice - one node, zero prompt engineering. Just budget an evening for the install and keep your expectations at "digital presenter," not "deepfake cinema." When you outgrow it, the bigger AIFSH packs (SeedV-class) are the step up; MiniMates is the light one you reach for when you just want the video.