Timestamped Lyrics Extractor (stable-ts)

Music videos run on a simple rule: the scene changes when the lyric changes. So the first thing every VRGameDevGirl music-video workflow needs is a timestamped transcription of the song - which line is sung when, and where the instrumental gaps are. That's this node's entire job, and it's the one that separates a real music video from a slideshow set to a beat. It's built on stable-ts, the timestamp-precise fork of Whisper that the community uses when plain Whisper's word-level timing isn't good enough.

The node takes an AUDIO input (feed it whatever audio you loaded for the project), runs stable-ts against it, and returns a single timestamped_lyrics_json string - structured timing data that the rest of the pack (the Video Editor, the scene planner) consumes as the backbone of the project. This is the "analyzes the timing, creates scene prompts from lyrics" step the author describes in the workflow walkthrough.

The inputs that matter

• model_name (default large-v3) - which Whisper checkpoint stable-ts uses. large-v3 is the accurate one; if transcription is crawling or VRAM is tight, a smaller model is a legitimate trade.
• language - set it to the song's language instead of auto if auto-detection ever drifts, which it does on instrumental intros.
• reference_lyrics - here's the party trick: paste the actual lyrics you wrote. When you supply them, the node can align against the known text instead of trusting Whisper's transcription, which fixes misheard lines before they ever become scene prompts.
• segment_mode - how output segments are built: whisper_chunks (raw Whisper segments), reference_lines / exact_reference_lines (align to your pasted lyrics, with or without strict line matching), or reference_stanzas / reference_scene_words (group into stanzas or scene-sized chunks). reference_stanzas is usually the sweet spot for music videos - scene-sized segments, not line-by-line.
• include_instrumental_gaps (default on) with instrumental_text - marks non-vocal stretches as [instrumental] instead of silence, and min_gap_seconds controls how long a pause has to be before it counts as a gap.
• min_scene_seconds / max_scene_seconds - clamp scene lengths; default 2–10s is a sane music-video range.
• vocal_tail_padding_seconds (default 0.6) - bleeds a little extra time onto the end of each sung segment so the video doesn't cut on the very last syllable.

Wiring it

The output is a string of JSON, so don't expect an image tensor out of it. It feeds the project's timing/lyric store - in the Video Builder that's the lyric mapping step where you review the extractor's output before the scenes get built. If a segment looks wrong, you can fix it in the editor rather than re-running the whole transcription, which is why the pack keeps transcription and scene-building as separate steps.

Install and gotchas

Standard pack install (Manager → search vrgamedev, or clone and pip install -r requirements.txt). stable-ts downloads its Whisper weights on first run, so the first transcription is slower than it has any right to be - that's just the model downloading.

The two real-world gotchas: instrumental intros - the author's own walkthrough notes that a lyricless intro causes issues, and if you're feeding an instrumental track, either set include_instrumental_gaps and give it a label, or plan a B-roll scene for that stretch. And reference lyrics that don't match the audio (wrong version, live vs studio) will make exact_reference_lines produce garbage; drop back to whisper_chunks or reference_stanzas if alignment goes weird. The node doesn't invent lyrics - it can only align to what it's given.