Music · Whisper-Passagen zu Zeilen

Lyrics are lines. Transcriptions are a paragraph. Everything you want to do afterwards - measure the form, copy the rhyme scheme, generate a cover that fits the melody - depends on the line breaks being right, and text heuristics are bad at it. The author measured this on a Toto transcript: a phrase-frequency approach broke mid-line ("hold me It's not in the way You say") because the most common word sequence is not the sung line. Which makes the rhyme scheme random, which was the whole point of measuring.

Whisper knows better, because Whisper gives you timestamps. One passage is usually one sung line. That's the entire premise of this node.

What it takes and what it gives

whisper_chunks is a WHISPER_CHUNKS object - that's comfy_mtb's Whisper output, not a string. Two things go wrong here if you don't know them.

The mtb chunks carry BPE tokens, not clean words. You get ĠIt 's Ġnot Ġin Ġthe Ġway and Ġtreat in ', where Ġ marks a word start and a token without it belongs to the previous word. The node reassembles them before anything else touches the text - without that, the syllable counter counts fragments and every rhyme word is wrong. So Ġtreat in ' becomes treatin', and the rhyme compares against something real.

Do not use Process Whisper Output's string output. Same flowing text as before, no timestamps. The docstring says it outright. If you're using this node, you want the chunk object.

The outputs are just lyrics and info. lyrics is the line-broken text with section marks in it, ready to feed FVM_LyricsProsodyAnalyze - that's the pairing this node exists for.

The three dials

pause_zeile (default 0) decides whether passages merge into lines. Leave it at 0 and every passage stays its own line, which is the right default: Whisper has already segmented, and passages sit back to back without a pause far more often than not. Raise it and the node glues passages together whose gap is shorter than the threshold. Fragments under two words get attached to the previous line regardless - that's a floor, not a setting.

pause_sektion (default 1.6 s) is the section boundary: a longer silence starts a new section. mit_sektionen (on by default) decides whether the section marks get written into the text at all.

Section detection also uses repetition, not just silence. Lines that repeat are the chorus, and the comparison runs on a normalised form (lowercase, punctuation stripped) because Whisper writes the same line slightly differently each time. A section needs at least two lines to count, otherwise you get single-line "sections" that don't describe any form.

Install

ComfyUI Manager, search comfyui-fvmtools, or:

cd ComfyUI/custom_nodes
git clone https://github.com/ping1979ping/comfyui-FVMtools

Restart. The node is under Add Node → FVMtools/music. Pure string processing - no models, no extra Python packages beyond what ComfyUI already has.

You will, however, need comfy_mtb for the WHISPER_CHUNKS type in the first place, plus a Whisper model through it. That's the real dependency here, and it's a separate pack.

Common issues

The input won't connect. You're trying to plug a STRING into a WHISPER_CHUNKS socket. Find the chunk output, not the text one.

Words are glued together or split oddly in the output. Check whether your chunks really carry the Ġ markers. If the text arrives pre-cleaned, the node's token reassembly is a no-op and everything still works - but if you see stray Ġ characters in your lyrics output, you fed it something that only looks like chunks.

Every line became one section. Raise pause_sektion a little, or turn mit_sektionen off and keep the plain lines. In dense songs with short instrumental gaps, 1.6 s is already aggressive.

The whole thing produced two giant lines. Chunks are only as good as the transcription. Long passages mean Whisper didn't segment - at which point the passage-per-line assumption is broken and you're back to text heuristics, where FVM_LyricsProsodyAnalyze's fliesstext segmentieren mode is the fallback.