VQ Text-Video Alignment

You typed "a black cat walking through rain" and got... a cat, then a cat-ish blob, then something that's mostly rain with cat vibes. That's prompt drift - the video starting faithful and slowly forgetting the prompt - and it's one of the most common ways long text-to-video generations fail. This node is a detector for exactly that: it scores how well your video matches the text prompt, frame by frame, and tells you whether the match is decaying over time.

No reference video needed - just your output and the prompt you fed the model. That makes it a staple for text-to-video evaluation, where a reference doesn't exist. It's also the node to reach for if you suspect your model "forgets" concepts late in the clip.

How it works

CLIP (ViT-B/32, auto-downloaded from HuggingFace on first use) encodes text and images into a shared space, so you can ask directly: "how similar is this frame to this prompt?" The node samples frames uniformly across the video - sample_frames of them, 8 by default - and measures the CLIP similarity between each frame and your prompt, mapped to a 0–1 scale. The overall alignment_score is the average. Then it looks at the trend: the difference between the last sampled frame's score and the first. Drop more than about 0.1 and the summary flags "temporal drift detected" - the video is losing the prompt over time.

Inputs and outputs

Required:

• video (IMAGE) - your frame tensor.
• prompt (STRING, multiline) - the exact prompt you want to check against. It's compared literally, so minor phrasing changes shift the number.

Optional:

• sample_frames (INT, default 8, range 1–32) - how many frames to score. More frames = better drift detection, slower run.

Outputs:

• alignment_score (FLOAT) - 0–1, higher means the video matches the prompt. Above ~0.6 is decent, below 0.4 is weak.
• summary (STRING) - report with the prompt, the score, and the drift message. Wire to a text display.

Installing

It ships with ComfyUI-VideoQuality-Metrics:

cd ComfyUI/custom_nodes
git clone https://github.com/jajos12/ComfyUI-VideoQuality-Metrics
pip install -r ComfyUI-VideoQuality-Metrics/requirements.txt

Restart ComfyUI; it's under Video Quality/CLIP. ComfyUI Manager installs the pack if you search "VideoQuality Metrics".

Common issues

• Empty prompt returns 0.0 with a warning. It checks for a blank prompt and bails. Type something.
• First run downloads ~600 MB of CLIP weights. Expect the first execution to hang while that happens.
• The drift warning only appears with 3+ sampled frames - with fewer, there's no trend to detect. Keep sample_frames at 8 or higher if you care about drift.
• Alignment is literal, not semantic. CLIP is good, but "walking through rain at night" will score differently than the same scene described differently. Compare like-for-like phrasing across runs, and don't expect the absolute value to be a perfect proxy for "did I get what I asked for."