Volcengine video to video lip sync

What it is

KIE_Next_volcengine_video_to_video_lip_sync_77b9f652 takes a video of a person and an audio clip, and returns the same video with the mouth moving to that audio. It's Volcengine's (ByteDance's cloud) lip-sync service, reached through KIE.ai's marketplace, wrapped as a single ComfyUI node with a VIDEO input, an AUDIO input and a VIDEO output.

This is the API route to the thing people usually solve locally with audio-driven avatar models - talking-head generation from a driving voice. Local options exist and they're decent, but they're a whole dependency stack, and they want a GPU that's already busy. If what you need is "this shot, this voice, done", paying per call is a reasonable trade, and the node means the result lands in your graph ready for the rest of your edit.

The honest caveats, same as any API node: your footage leaves your machine, and your KIE credit balance absorbs the cost.

How it works

The node assembles the request, submits it to KIE's task queue, then waits for it, polling at 2.5-second intervals and backing off toward 15 until the job resolves or timeout_seconds (1200 by default, up to 7200) expires. On success it downloads the result and hands back a native ComfyUI VIDEO object. The model travels as KIE's marketplace id for Volcengine's lip-sync endpoint.

The nice part is that media handling is automatic. The video_url and audio_url fields are real VIDEO and AUDIO sockets, so you can wire a loaded video and a TTS output straight in - the pack uploads both to KIE for you and substitutes the URLs. No hosting, no copy-pasting links.

The inputs

• mode - required, and worth understanding before you queue: lite is for single-person frontal footage and is the faster option; basic is for single-person complex scenes and adds scene segmentation and speaker identification. Start on lite.
• video_url - required, a VIDEO socket. Documented limits matter here: 360p to 1080p (above 1080p gets compressed down, below 360p isn't accepted), MOV/MP4/HDR with H.264 recommended, 24–60 fps, max 500 MB. Other formats or codecs may get transcoded behind the scenes.
• audio_url - required, an AUDIO socket, and the docs describe it as pure vocal audio: mpeg, wav, aac, mp4 or ogg, max 10 MB. A track with music in it is not what this wants.
• separate_vocal - false by default. Turn it on and the service tries to suppress background noise in the audio before driving the mouth with it.
• open_scenedet - false by default, usable only in basic mode. Scene segmentation and speaker identification.
• align_audio - true by default, and this one is a real editorial decision: in lite mode it loops the video when the audio is longer than the picture, instead of letting the audio run past the end.
• align_audio_reverse - false by default, lite only, and it needs align_audio on to mean anything: loops the video backwards instead of forwards.
• templ_start_seconds - 0 by default, lite only: a start offset into the template video, in seconds.

Outputs

video - the lip-synced clip as a native ComfyUI video. url - the primary result URL. all_urls_json - every URL the response contained. task_id - the job handle, handy if you want to poll or keep a record. raw_json - the full provider response, where failure reasons live. credits_consumed and credits_left - what the call cost.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/felipederosilva/ComfyUI-KIE-Nodes-Next
cd ComfyUI-KIE-Nodes-Next && pip install -r requirements.txt

Restart ComfyUI. Dependencies are requests, Pillow, numpy, PyYAML - no models to download and no local inference at all. There's a one-click Windows installer in the repo's dist/ folder for anyone who'd rather skip the shell. ComfyUI Manager support is where the project is headed (the repo carries registry metadata under publisher felipeosilva); the clone is the install that works now.

Paste your KIE.ai key once at Settings → KIE.ai Nodes Next → Connection - validated before it replaces a working key, shown only as a masked fingerprint. KIE_API_KEY in the environment overrides the saved key on headless installs.

What'll bite you

Two people in frame. Both modes are documented as single-person; the extra face confuses the driver.

Music instead of a voice. The audio side wants clean vocals, and 10 MB is a hard ceiling. Run a TTS output or a separated vocal stem in, not a full mix - the stem separation node in this same pack is a sensible upstream.

Odd footage. Under 360p, over 1080p, or an exotic codec: expect a transcode, a compression, or a rejection. Pre-transcoding to H.264 MP4 in the 720p–1080p band is the cheap way to remove a variable.

A timeout that looks like a failure. These jobs can outlast the default wait. Check task_id with KIE • Task Status or KIE • Wait for Task before resubmitting, because a resubmit is another paid call.