MIDI Lyrics Alignment

MIDI Edit Lyrics is the old workhorse; this is its deterministic, rebuild-the-world sibling. Where Edit Lyrics preserves the original phrasing and does careful surgery inside each section, Alignment throws the original <SP> structure away and rebuilds it around your new lyric's natural sentence boundaries. That makes it the right tool when your new words have nothing structurally in common with the original - different sentence count, wildly different length, or you just want the pauses to land where you would breathe.

How it works

The author's description is "sequential mapping + greedy compression + CT-Transformer smart segmentation," which sounds like a lot until you see the steps:

• Segment first. New lyrics split on punctuation; anything longer than 10 characters gets punctuation restored by the CT-Transformer model, then split. Segmentation only looks at your lyric - it never tries to match the original song's sentence count.
• Rebuild the SP structure. All original <SP> markers are discarded, then re-created around the segmented lyric: [SP] sentence [SP] sentence [SP].
• Sequential mapping (new characters ≤ original tokens). The first C non-SP tokens each carry one character in order; leftover tokens are dropped. Each character inherits its token's duration, pitch, and f0.
• Greedy compression (more characters than tokens). Characters get distributed by token duration proportion - long tokens carry more words, short tokens fewer. Split groups divide duration evenly with a 0.1-second floor.
• Rebuild f0 per token and recompute time from frame counts (50 fps internally) to kill cumulative rounding error. The changelog notes this specifically avoids SoulX-Singer's cryptic "could not broadcast" error.

One thing to internalize: total duration is not conserved here. SP count and SP duration both change with your lyric, so the output track is longer or shorter than the original. That's by design, not a bug.

Inputs and outputs

• midi_json - the track JSON.
• lyrics - your new text (multiline, mixed Chinese/English fine, punctuation and newlines welcome).
• normalize_digits (default ON) - Arabic digits become Chinese numerals.
• force_tone4 (default OFF) - force the fourth tone on high notes.
• speed (0.1–3.0, default 1.0).

Two outputs:

• midi_json - hand this to MIDI Synthesize Audio.
• warnings - a STRING that's empty when all is well. Two warning types: HIGH_SPLIT_RATIO(chars=C,slots=nonsl) means your character count badly overshoots the song's token count, and MIN_DURATION_UNRESOLVED means some character ended up with under 0.1s of duration and nothing to borrow from. Both mean the output was produced but the singing may suffer - shorten your lyric or pick a better-matching song. Warnings carry a (t{idx}) prefix so you know which track they came from.

There's a nice multi-track behavior: lyrics are distributed across tracks in proportion to each track's non-SP duration, so a 30-second track gets most of the words and a 1.5-second track gets almost none. Tracks that end up with no lyrics are kept untouched.

Which one should you use?

Think of it as Edit Lyrics for conservative edits, Alignment for wholesale rewrites. If you're changing a few words and want the song's phrasing untouched, Edit Lyrics is stable and battle-tested. If you're writing a completely new verse and want natural segmentation, Alignment is the better fit - and it's the newer code, which means it's been getting the bulk of recent fixes.

Install is shared with the pack: Manager ("ComfyUI MIDI Edit") or git clone --recursive, then pip install -r requirements.txt. No SoulX models needed just to run alignment - the CT-Transformer punctuation model (~270 MB) and NLTK data auto-download to ComfyUI/models/ on first use.