Nodes/Comfyui-zhenzhen/Zhenzhen_lip_sync
ComfyUI Node

Zhenzhen_lip_sync

Give Your Generated Clip a Voice, Text or Audio

By T8mars·Created 12 months ago·Updated 4 days ago· 724
Zhenzhen_lip_sync
    • video
    • video_url
    • task_id
    • response
    video_id
    task_id
    modetext2video
    text
    voice_languagezh
    zh_voice阳光少年
    en_voiceSunny
    voice_speed1.0
    seed0
    api_key
    video_url
    audio_typefile
    audio_file
    audio_url
    skip_errorfalse

    You generated a Kling video of a person talking... but they're not talking. This node fixes that. It's Kling's lip-sync, which takes a video you already made, adds a voice, and makes the mouth move in sync with it. Two input modes: text2video, where you type the line and the API picks a TTS voice, and audio2video, where you supply your own audio file and the model syncs the lips to that.

    The author's own notes are refreshingly honest about it: the effect is "okay-ish" and the speed is "so-so." Which is accurate - this is not a $50-a-month dedicated avatar tool, it's Kling's lip-sync bolted onto your existing generation. But for "my character should say this one line" it's remarkably useful, and it works in both Chinese and English, which most of this ecosystem's tools can't claim.

    How it works

    The important quirk: this node doesn't generate a fresh video. It consumes one. You give it the video_id and task_id from a previous Kling generation (the outputs you were told to keep from the text2video / multi-image nodes - this is why), or a direct video_url. Then it runs TTS + lip-sync through the proxy and returns a new video with the mouth moving.

    The inputs that matter

    • video_id / task_id - the two strings from your earlier Kling node run. Get these wrong (or leave them empty and only use video_url) and the node has nothing to sync.
    • mode - text2video (type the line, pick a voice) or audio2video (bring your own audio).
    • text - the line to speak. Only used in text2video mode.
    • voice_language - zh or en. Pick before picking a voice, obviously.
    • zh_voice - 35 Chinese voices, and they're genuinely characterful: everything from 阳光少年 (sunny youth) to 东北老铁 (Northeastern bro) to 刀片烟嗓 (raspy smoker). This is the most fun dropdown in the pack.
    • en_voice - 27 English voices, from Sunny to Siren.
    • voice_speed - 0.82.0, default 1.0.
    • audio_type / audio_file / audio_url - for audio2video mode: file or url, then the path or link. The file form takes a local path string.
    • seed - lock a take.

    Outputs: video (VIDEO), video_url (STRING), task_id (STRING), response (STRING).

    Install

    Manager → Comfyui_Comfly, or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/ainewsto/Comfyui_Comfly
    

    Restart. No model downloads; requirements.txt is the standard aiohttp/requests/Pillow/numpy set. Key from ai.comfly.chat in Comflyapi.json or the node's api_key.

    Where people get burned

    The biggest one is architectural: people drop a brand-new Comfly_kling_text2video node in, expect lip-sync output, and get nothing - because this node needs the already-generated video's ids, not a fresh prompt. Also, the speaking-face quality ceiling is real; keep the character's mouth relatively visible in the source clip and the effect lands much better than a tiny distant face. And keep your line short - long monologues drift off-sync, and the author's own "so-so" speed rating means every extra second is more render time. Oh, and the voice dropdowns are Chinese-first: the EN voices are there, but the zh list is where the personality lives.

    Categoryzhenzhen/Comfly_kling

    Inputs (15)

    NameTypeDefaultDescription
    video_idSTRING
    task_idSTRING
    modeCOMBOtext2video2 options: text2video, audio2video
    textSTRING
    voice_languageCOMBOzh2 options: zh, en
    zh_voiceCOMBO阳光少年35 options: 阳光少年, 懂事小弟, 运动少年, 青春少女, 温柔小妹, 元气少女, +29
    en_voiceCOMBOSunny27 options: Sunny, Sage, Ace, Blossom, Peppy, Dove, +21
    voice_speedFLOAT1.00.8–2
    seedINT00–2147483647
    api_keyoptSTRING
    video_urloptSTRING
    audio_typeoptCOMBOfile2 options: file, url
    audio_fileoptSTRING
    audio_urloptSTRING
    skip_erroroptBOOLEANfalse开启后,节点失败时不报错、按旧行为返回默认空结果;关闭时(默认)失败直接抛出错误。

    Outputs (4)

    NameTypeDescription
    videoVIDEO
    video_urlSTRING
    task_idSTRING
    responseSTRING