ComfyUI-MMS-Aligner

A ComfyUI custom node that force-aligns an audio clip with its text transcript and writes a timestamped SRT subtitle file.

Uses Meta's facebook/mms-1b-fl102 model (102 languages, FLEURS-trained) via transformers + torchaudio.functional.forced_align. Pure Python — no C++ compilation, no MSVC, no extra build tools.

Why FL102?

facebook/mms-1b-fl102 is the only MMS model used by this node. It covers 102 languages, trained on the cleaner FLEURS dataset, with per-language adapters (each language uses its own native tokenizer — no romanization required for the common cases).

If your language isn't in FL102, this node won't work for you — but for the languages most people actually need (English, Vietnamese, Chinese Mandarin, Spanish, French, German, Japanese, Korean, Arabic, Hindi, Russian, etc.), it does.

Screenshots

Features

• One node: audio + text in → SRT out (string and/or file).
• Single model, auto-downloaded to ComfyUI/models/mms/facebook__mms-1b-fl102/ on first use (~3.2 GB).
• Pure-Python install: no compiler, no headaches on Windows.
• Language: ~30 common languages in a dropdown, any ISO 639-3 code via custom, or auto-detect from the text.
• Optional romanize fallback (uroman) for edge cases.
• Six segmentation modes: single-word (karaoke-style), max chars, punctuation, newlines, or combinations.
• Optional cue-gap bridging (gap_ms) to eliminate sub-300 ms flicker between consecutive cues.
• Optional output splitting (split_count, up to 5) to emit name_1.srt, name_2.srt, … for chunked workflows.
• Tooltips on every input.
• All processing in-memory — no temp WAV files written.

Installation

Manual

cd ComfyUI/custom_nodes
git clone https://github.com/idealweek/ComfyUI-MMS-Aligner
cd ComfyUI-MMS-Aligner
pip install -r requirements.txt

Restart ComfyUI. The node appears under audio/alignment as "MMS Audio-Text Aligner (SRT)".

Requirements: torch and torchaudio are already provided by ComfyUI. The extras (transformers, huggingface_hub, uroman, lingua-language-detector) are pure Python.

Usage

1. Add a LoadAudio node and connect its output to the aligner's audio input.
2. Type or wire the transcript into the text input.
3. Pick language and segmentation_mode (use single_word for karaoke-style one-word-per-cue output).
4. Set output_name (no extension — .srt is added automatically). Files are written into ComfyUI's output directory. The srt string output is always populated.
5. (Optional) raise split_count (max 5) to split the result into output_name_1.srt, output_name_2.srt, …
6. (Optional) tune gap_ms to bridge tiny silences between consecutive cues — handy for single_word mode to prevent flicker.

Example — Vietnamese

| Input | Value | |---|---| | audio | (from LoadAudio) | | text | Xin chào, đây là một ví dụ.\nMột dòng nữa. | | language | Vietnamese | | romanize | False | | segmentation_mode | max_chars+punctuation+newlines | | max_chars_per_line | 42 | | gap_ms | 300 | | output_name | subs | | split_count | 1 |

Output (srt):

1
00:00:00,120 --> 00:00:02,340
Xin chào, đây là một ví dụ.

2
00:00:02,500 --> 00:00:03,820
Một dòng nữa.

Inputs

| Field | Type | Default | Description | |---|---|---|---| | audio | AUDIO | — | ComfyUI audio. Resampled to 16 kHz mono internally. | | text | STRING (multiline) | "" | Transcript. | | language | enum | auto | auto, ~30 common languages, or custom. | | custom_language_code | STRING | "" | ISO 639-3 (used only when language=custom). | | romanize | BOOL | False | Optional fallback. Usually leave off — the per-language tokenizer handles native scripts. | | segmentation_mode | enum | max_chars+punctuation+newlines | See below. | | max_chars_per_line | INT | 42 | 0 disables the cap. | | remove_punctuation | BOOL | False | Strip .,!?;:"()[]{}—–…«»“”‘’ from SRT lines (keeps apostrophes/hyphens). Doesn't affect segmentation. | | gap_ms | INT | 300 | Bridge consecutive cues whose gap is under this many ms by snapping prev.end to next.start. Real pauses (≥ gap_ms) are preserved. 0 disables. | | output_name | STRING | "subtitles" | Base file name (no extension) written to ComfyUI's output dir. .srt is appended automatically; path separators are stripped. | | split_count | INT | 1 | 1 = single file. 2–5 splits cues into near-equal chunks named <output_name>_1.srt, <output_name>_2.srt, … |

Segmentation modes

• single_word — one word per cue (karaoke-style). Pair with gap_ms to avoid flicker.
• max_chars — pack words greedily up to max_chars_per_line.
• punctuation — break at . ! ? , ; :.
• newlines — break wherever the input text has a line break.
• max_chars+punctuation — both.
• max_chars+punctuation+newlines — all three.

Outputs

• srt (STRING) — full SRT content (un-split, even when split_count > 1).
• output_path (STRING) — newline-separated list of paths written to disk (one entry when split_count=1, N entries otherwise).

How it works

1. Resample the input audio to 16 kHz mono in memory.
2. Detect the language from the transcript (lingua) when language=auto.
3. Download facebook/mms-1b-fl102 to ComfyUI/models/mms/ on first run (cached after).
4. Load the model with the per-language adapter (model.load_adapter(lang) and processor.tokenizer.set_target_lang(lang)).
5. Tokenize each transcript word with the language's native tokenizer (optionally romanize first).
6. Get emissions via a single forward pass; convert to log-probs.
7. Force-align with torchaudio.functional.forced_align; merge token spans into word boundaries.
8. Segment word timings into SRT cues per the chosen mode.
9. Emit the SRT as a string and (optionally) write to disk.

Troubleshooting

• "Could not auto-detect language" — text too short or mixed; set language explicitly.
• "Transcript produced no alignable tokens" — text contains characters not in the language's tokenizer vocab. Try romanize=True, or check the language code.
• Slow first run — model is downloading (~3.2 GB); subsequent runs use the cache.
• Wrong language adapter — make sure the ISO 639-3 code matches one of the FL102 languages.

License

MIT

Credits

• Meta MMS — facebook/mms-1b-fl102
• transformers — model loading & adapters
• torchaudio.functional.forced_align — alignment math
• uroman — pure-Python universal romanizer (ISI)
• lingua-language-detector — text language detection