ComfyUI Node

lipsync-2

A lipsync node that actually takes your audio

By Runware·Created 2 years ago·Updated about a month ago· 140
lipsync-2
    • video
    video
    audio
    numberResults1
    providerSettings.sync.activeSpeakerDetection.autoDetectfalse
    providerSettings.sync.occlusionDetectionEnabledfalse
    safetyfalse
    safety.checkContentfalse
    safety.modefast
    providerSettings.sync.syncModebounce
    providerSettings.sync.temperature0.50
    ttlfalse
    ttl_value60
    outputFormatMP4
    outputQuality95
    advanced_json

    Lipsync is one of those jobs the community talks about far more than it builds, because every serious local option is a dependency rabbit hole - face detection, Wav2Lip weights, audio-pipeline glue, half of it breaking on every ComfyUI update. This node skips all of it: give it a video and an audio track, and lipsync-2 runs on Runware's cloud and hands back a VIDEO where the mouth actually matches the words. Same input, no local pipeline to babysit.

    The two inputs that are the whole job

    • video (required, STRING): the clip whose face should speak, as a UUID or URL.
    • audio (required, STRING): the track to sync to, also a UUID or URL. You're not feeding a waveform tensor here - the cloud wants a reference to the file, so drop a URL or an uploaded media UUID in.

    Everything else is quality-of-life. The interesting ones:

    • providerSettings.sync.syncMode: what happens when the audio and video lengths don't match - bounce, loop, cut_off, silence, remap. This is the "my clip is 6 seconds and the VO is 9" decision, and it's the one setting people usually want to touch.
    • providerSettings.sync.temperature (default 0.5, range 0–1): how expressive the facial movement is. Lower is steadier and more conservative; higher gets livelier mouths and eyebrows, with a side of "too much."
    • providerSettings.sync.activeSpeakerDetection.autoDetect: find and target whoever is actually talking. Turn this on for multi-person shots.
    • providerSettings.sync.occlusionDetectionEnabled: handles obstructed faces - the classic hand-in-front-of-mouth or microphone problem.

    There's an advanced_json escape hatch that the tooltip tells you is for providerSettings.sync.segments - the manual "this chunk of audio maps to this chunk of video" mapping, for when auto-detect isn't enough.

    How it works

    Same pack machinery as every Runware node: your inputs ride up over REST, lipsync-2 does the sync on cloud GPUs, and the pack downloads the result and wraps it as a native VIDEO for the rest of your graph. The safety group (checkContent, mode of none/fast/full) and ttl/outputFormat/outputQuality widgets are the standard shared block. Cost per run appears on the title bar.

    Install and honest gotchas

    Install the pack once - ComfyUI Manager, search Runware - or git clone https://github.com/Runware/ComfyUI-Runware into custom_nodes plus pip install -r requirements.txt, then set a Runware API key (Settings → Runware API key, RUNWARE_API_KEY, or runware auth login).

    Two things to know before you queue. First, expect to tune syncMode and temperature on your first clip; the defaults are safe, not tailored to your footage. Second, this is a paid API - a multi-speaker clip with numberResults cranked up is a bill you'll see, so iterate on one result before you roll several. And if your source face is half-hidden, turn on occlusion detection before you blame the model; that's exactly what it's for.

    CategoryRunware/Video/sync

    Inputs (15)

    NameTypeDefaultDescription
    videoSTRINGVideo input (UUID or URL).
    audioSTRINGAudio input (UUID or URL).
    numberResultsoptINT11–4Number of results to generate. Each result uses a different seed, producing variations of the same parameters.
    providerSettings.sync.activeSpeakerDetection.autoDetectoptBOOLEANfalseAutomatically detect and target the active speaker.
    providerSettings.sync.occlusionDetectionEnabledoptBOOLEANfalseEnable occlusion handling for obstructed faces.
    safetyoptBOOLEANfalseEnable to set safety. Off uses the model's default.
    safety.checkContentoptBOOLEANfalseEnable or disable content safety checking.
    safety.modeoptCOMBOfastSafety checking mode for video generation.
    providerSettings.sync.syncModeoptCOMBObounceSynchronization strategy when audio and video durations don't match.
    providerSettings.sync.temperatureoptFLOAT0.500–1Expressiveness of lip sync and facial movements.
    ttloptBOOLEANfalseEnable to set ttl. Off uses the model's default.
    ttl_valueoptINT60Time-to-live (TTL) in seconds for generated content. Only applies when `outputType` is `URL`.
    outputFormatoptCOMBOMP4File format for the generated video.
    outputQualityoptINT9520–99Compression quality of the output. Higher values preserve quality but increase file size.
    advanced_jsonoptSTRINGOptional JSON merged into the request. For: providerSettings.sync.segments

    Outputs (1)

    NameTypeDescription
    videoVIDEO