SyncReward Score

A few years of workflow-building taught us to eyeball pictures and move on. Sound broke that habit. Once video models started inventing audio - LTX-2.3 in the same generation pass, Ovi and Seedance like that, or a foley stage with MMAudio bolted onto a silent Wan render - "does the clap land on the hands" became a real question, and eyeballing it means watching every candidate at half speed. SyncReward trained a model on human ratings of exactly that: how well the audible events line up with the visible ones. SyncReward Score gives you the number, 0 to 2.

It's hiroki-abe-58's unofficial packaging of a research model (WeChatCV/SyncReward, Renmin University + WeChat), with the official code vendored and run in-process. No API, no key.

How the number gets made

The model doesn't look at your whole clip. It cuts a centered 4.75 s window out of it - 18 segments of 0.5 s each, overlapping by 0.25 s - because that's the shape its config.json was trained on. Each segment goes through Meta's PE-AV encoder (audio and video towers), the Stage-2 head compares them under bfloat16 autocast, and out comes a prediction of a human rating. If your clip is shorter than the window, the last frame and the tail of the audio get repeated to fill it, which is worth knowing if you're scoring 2-second loops.

Practically: about 10 s to load once, then roughly 1.6–2 s per 5-second clip on an RTX 5090, peaking near 9.8 GiB. Next to a video render, that's a rounding error.

Inputs and outputs

Three required inputs. model comes from the Loader. video is a VIDEO - and it needs an audio track, which means a Load Video whose file actually has sound. encode_batch_size (1–36, default 18) is purely a VRAM dial: 18 is the official setting and costs 9.8 GiB, 1 costs 6.7 GiB. Lowering it can nudge the reward by one bfloat16 step (0.008) on some clips, so pick a value and stick with it if you're comparing clips against each other.

The optional audio input is the one people miss. Hook an AUDIO in and the video's own track is ignored - the model scores that sound against the picture instead. That's how you use it as a foley checker: generate the picture, generate sound separately with MMAudio, score the pairing.

Two outputs. reward is a plain FLOAT; wire it wherever numbers go, or into SyncReward Unload if you want the unload to happen after scoring. report is a JSON STRING with more than the number: reward, clip lengths, fps, frame count, a shorter_than_window flag, scoring_seconds, peak VRAM and which checkpoint ran. Send it to a Show Text node - that flag alone explains a lot of weird scores.

What the score will and won't tell you

This is the part the README screams about, and it's right to. On the official benchmark the model reproduces the paper closely - Spearman 0.781, pairwise accuracy 0.835 against human ratings on 1,000 clips. So it agrees with people on average. But on controlled edits to well-synced clips:

• Moving the audio by up to a second changes the reward by about 0.01. It is not a timing detector. If you need "is this 200 ms late," this is the wrong tool and no amount of prompting will fix that.
• Silence, noise, or another clip's audio only lowers the score in roughly 30–70% of clips. It's not a missing-sound test either.
• Anything below about 0.1 between two scores is noise - that's the spread you get from changes that shouldn't matter at all.
• Encoding moves it: a heavy re-encode, half resolution or 16 kHz audio each drop the reward by 0.14–0.21, which is 55–80% of what removing the sound does.

So use it as a coarse sanity check on a batch, not as a pass/fail gate. "This one is clearly better synced than that one" is fair when the gap is large. "This one is 0.04 better" is not.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-SyncReward
python -m pip install -r ComfyUI-SyncReward/requirements.txt
python custom_nodes/ComfyUI-SyncReward/tools/download_models.py --models-dir ComfyUI/models/syncreward

Or search "SyncReward" in ComfyUI Manager. Either way, the ~15 GB of weights go in models/syncreward, and ComfyUI has to be new enough for the V3 node API (built against v0.38.0). Restart, then load a graph from workflows/gui/.

Things that trip people up

"The video has no audio track." That's the real error text, and it's literal - the scorer refuses rather than scoring silence. Either use a file with sound or feed the optional audio.

"The clip is too short to score." Needs 2 frames and 0.1 s of audio - usually an upstream node handing over an empty batch.

Trimmed videos score differently. An untouched Load Video is decoded from the file exactly as the official code does it; a trimmed, cropped or image-built video goes through ComfyUI's own frames, and ComfyUI uses a different colour matrix for untagged YUV - worth up to about 0.07 of reward. When comparing numbers, feed untrimmed files.