ComfyUI Node

lipsync-2

A lipsync node that actually takes your audio

By Runware·Created 2 years ago·Updated 16 days ago· 140
lipsync-2
    • video
    video
    numberResults1
    providerSettings.sync.activeSpeakerDetectionfalse
    providerSettings.sync.occlusionDetectionEnabledfalse
    safetyfalse
    safety.checkContentfalse
    safety.modefast
    providerSettings.sync.syncModebounce
    providerSettings.sync.temperature0.50
    ttlfalse
    ttl_value60
    outputFormatMP4
    outputQuality95
    advanced_json

    Lipsync is one of those jobs the community talks about far more than it builds, because every serious local option is a dependency rabbit hole - face detection, Wav2Lip weights, audio-pipeline glue, half of it breaking on every ComfyUI update. This node skips all of it: give it a video and an audio track, and lipsync-2 runs on Runware's cloud and hands back a VIDEO where the mouth actually matches the words. Same input, no local pipeline to babysit.

    The two inputs that are the whole job

    • video (required, STRING): the clip whose face should speak, as a UUID or URL.
    • audio (required, STRING): the track to sync to, also a UUID or URL. You're not feeding a waveform tensor here - the cloud wants a reference to the file, so drop a URL or an uploaded media UUID in.

    Everything else is quality-of-life. The interesting ones:

    • providerSettings.sync.syncMode: what happens when the audio and video lengths don't match - bounce, loop, cut_off, silence, remap. This is the "my clip is 6 seconds and the VO is 9" decision, and it's the one setting people usually want to touch.
    • providerSettings.sync.temperature (default 0.5, range 0–1): how expressive the facial movement is. Lower is steadier and more conservative; higher gets livelier mouths and eyebrows, with a side of "too much."
    • providerSettings.sync.activeSpeakerDetection.autoDetect: find and target whoever is actually talking. Turn this on for multi-person shots.
    • providerSettings.sync.occlusionDetectionEnabled: handles obstructed faces - the classic hand-in-front-of-mouth or microphone problem.

    There's an advanced_json escape hatch that the tooltip tells you is for providerSettings.sync.segments - the manual "this chunk of audio maps to this chunk of video" mapping, for when auto-detect isn't enough.

    How it works

    Same pack machinery as every Runware node: your inputs ride up over REST, lipsync-2 does the sync on cloud GPUs, and the pack downloads the result and wraps it as a native VIDEO for the rest of your graph. The safety group (checkContent, mode of none/fast/full) and ttl/outputFormat/outputQuality widgets are the standard shared block. Cost per run appears on the title bar.

    Install and honest gotchas

    Install the pack once - ComfyUI Manager, search Runware - or git clone https://github.com/Runware/ComfyUI-Runware into custom_nodes plus pip install -r requirements.txt, then set a Runware API key (Settings → Runware API key, RUNWARE_API_KEY, or runware auth login).

    Two things to know before you queue. First, expect to tune syncMode and temperature on your first clip; the defaults are safe, not tailored to your footage. Second, this is a paid API - a multi-speaker clip with numberResults cranked up is a bill you'll see, so iterate on one result before you roll several. And if your source face is half-hidden, turn on occlusion detection before you blame the model; that's exactly what it's for.

    CategoryRunware/Video/sync

    Inputs (14)

    NameTypeDefaultDescription
    videoSTRINGVideo input (UUID or URL).
    numberResultsoptINT11–4Number of results to generate. Each result uses a different seed, producing variations of the same parameters.
    providerSettings.sync.activeSpeakerDetectionoptBOOLEANfalseAutomatically target the speaking face in a multi-person clip.
    providerSettings.sync.occlusionDetectionEnabledoptBOOLEANfalseEnable occlusion handling for obstructed faces.
    safetyoptBOOLEANfalseEnable to set safety. Off uses the model's default.
    safety.checkContentoptBOOLEANfalseEnable or disable content safety checking. Increases total generation time.
    safety.modeoptCOMBOfastSafety checking mode for video generation.
    providerSettings.sync.syncModeoptCOMBObounceSynchronization strategy when audio and video durations don't match.
    providerSettings.sync.temperatureoptFLOAT0.500–1Expressiveness of lip sync and facial movements.
    ttloptBOOLEANfalseEnable to set ttl. Off uses the model's default.
    ttl_valueoptINT60Time-to-live (TTL) in seconds for generated content. Only applies when `outputType` is `URL`.
    outputFormatoptCOMBOMP4File format for the generated video.
    outputQualityoptINT9520–99Compression quality of the output. Higher values preserve quality but increase file size.
    advanced_jsonoptSTRINGOptional JSON merged into the request. For: inputs.audio, providerSettings.sync.segments

    Outputs (1)

    NameTypeDescription
    videoVIDEO