Lip Sync Text

It's a text box with a wire on the end

"Lip Sync Text" is a misleading name. This node moves no lips. It takes a string in and gives the same string back - that's the whole implementation. What makes it worth having is where it sits: it ships inside the ComfyUI VRM Pose Editor 3D pack (ketle-man/comfyui-vrm-pose-editor), and it exists so one line of dialogue can feed two things at once - your TTS node, and the VRM mouth that has to move in sync with it.

The shape of the problem: the pack's PoseEditor3D node renders a posed VRM in the browser and hands the frame out as an IMAGE, its mouth animation comes from the pack's own lip sync, and the text that lip sync speaks lives in a lip_text field on that node. Audio and a matching mouth makes one sentence have two consumers, and typing it twice is how you get a render where the voice says one thing and the face says another. That's what a value node is for: one source, fanned out.

How it works (the whole file is 20 lines)

From lip_sync_text_node.py, minus the boilerplate:

class LipSyncTextNode:
    @classmethod
    def INPUT_TYPES(cls):
        return {"required": {"text": ("STRING", {"default": "", "multiline": True})}}

    RETURN_TYPES = ("STRING",)
    RETURN_NAMES  = ("text",)
    FUNCTION      = "output_text"
    CATEGORY      = "3D Pose"

    def output_text(self, text: str):
        return (text,)

No trimming, no normalization - newlines and all. The interesting work happens in the browser: js/lip_sync.js maps kana rows and common English vowel groups onto the five VRM vowel shapes (aa / ih / ou / ee / oh), then lays those units on a timeline at roughly 110 ms per unit. Timing is estimated from the characters; there's no phoneme alignment anywhere in this, so it looks right in short bursts and drifts across a long paragraph.

The input and output that matter

• text - STRING, multiline, default empty. The only input.
• text - STRING, the output. Yes, it's the input again.

Wire it to two places: whatever makes the voice (TTS Audio Suite, or the pack's own 🔊 Create audio from text button, which talks to a local Lemonade or VOICEVOX server), and the lip_text input on PoseEditor3D - so the same line is already in the editor's Lip Sync box when you open it.

One catch, and it's mechanical: the pack's extension hides the lip_text widget on the 3D Pose Editor node, so there's no box you can see to connect to. Right-click the node and use Convert Widget to Input → lip_text. If your frontend version won't offer a hidden widget there, there's a fallback that's arguably nicer: type the line into the editor's Lip Sync box, and closing the modal writes it back onto the node's lip_text value - which the node returns from its own lip_text STRING output. Feed that to your TTS node and you can skip this node's text box entirely.

If you generate audio through the pack's built-in TTS route instead of an external node, note the cap there: that endpoint rejects anything over 2000 characters. The node itself has no limit.

Install

ComfyUI Manager → Install Custom Nodes → search comfyui-vrm-pose-editor → Install → restart, which isn't a formality: the pack registers server routes on import.

Manual, if you'd rather:

cd ComfyUI/custom_nodes
git clone https://github.com/ketle-man/comfyui-vrm-pose-editor
# restart ComfyUI

Keep the folder name exactly comfyui-vrm-pose-editor. ComfyUI serves the pack's frontend scripts from /extensions/<folder name>/, and other tools load them from that literal path - rename it and the editor's buttons quietly stop working.

Then drop a Lip Sync Text node on the canvas from the 3D Pose category. No model files, no downloads, no Python dependencies for this node. The pack's only declared dependency is imageio-ffmpeg (for WebM → MP4); the rest is vendored JavaScript - three.js r160 and @pixiv/three-vrm 2.1.0 - running in your browser tab, not your Python environment.

Where people get burned

The text doesn't show up until you re-open the editor. The value is pushed into the Lip Sync box when you click Light or Pose on the node. If the modal was already open when you changed the text upstream, it won't update itself - close and reopen. That's the usual "why is it still speaking the old line".

The write-back fights your wire. Closing the modal copies whatever is in the Lip Sync box back onto the node's lip_text value. If you wired an input there, the wire wins on the next run and anything you hand-edited evaporates. Pick one source of truth.

Kanji get a fixed open mouth. Kana map to actual vowels; kanji have no reading a browser-side mapper can use, so each one is just an open-mouth unit. Write Japanese in kana if you care about shape accuracy.

A TTS engine that isn't running. Lemonade and VOICEVOX have to be up on the same machine, and the editor only accepts localhost / loopback URLs - no remote boxes. If Test connection fails, that's the whole story. Text-only lip sync (the ▶ Text button) needs neither installed. And VOICEVOX isn't bundled: its terms require a credit when you publish audio, plus per-character conditions, and the pack's MIT license doesn't cover it.

Modern-node UI quirks. The extension detects ComfyUI's Nodes 2.0 / Vue rendering and hides its widgets differently depending on which canvas you're on. If buttons or the lip_text conversion act oddly, that frontend rewrite is the usual suspect.