Whisper Segments Alignment Injector

Here's the classic dubbing/subtitle pipeline: you write a script, a TTS node reads it aloud, then you run Whisper over the audio to get timestamps. Problem: Whisper transcribes what it hears, not what you wrote. TTS voices garble, hallucinate, and Chinese is the worst offender - 引离 and 云璃 sound identical but are different characters. The result is subtitles that disagree with the voice, which defeats the entire exercise. Whisper Segments Alignment Injector is the node that fixes the text while keeping Whisper's timing. It's the reason this pack exists.

How it works. You feed it the segments_alignment value from a Whisper ASR node (the list of segments with start, end, and value) plus replace_input_str: your full original script as a string. For each segment it walks through the script using character counts, then runs an edit-distance-style dynamic programming alignment between the segment's Whisper text and the matching slice of your script. The clever part is the substitution cost, courtesy of the ASRMappingTextV3 mapper in the source: matching pinyin gets a cheap cost (0.4) versus a full mismatch (1.2), so Chinese homophones actually align instead of fighting. Punctuation is skipped entirely. It then writes the matched slice of your clean script into the segment's value. Output: Whisper's timing, your exact words.

Both inputs are force-input wires. replace_input_str is a plain STRING - wire in a text node holding the full script in the same order the audio plays. There's one output, segments_alignment, with the corrected text ready for the TimeFix and Merge nodes downstream.

Install. Part of teddy1565/ComfyUI-TTS2Whisper. ComfyUI Manager: search "ComfyUI-TTS2Whisper", install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/teddy1565/ComfyUI-TTS2Whisper

Restart, then make sure the dependencies actually installed. The requirements are pypinyin, numpy, torchaudio, torch. That pypinyin entry is the one people forget, and it matters here - the source imports it inside a try/except ImportError, and if it's missing while your text is Chinese, the pinyin lookup crashes the node. Run pip install -r ComfyUI/custom_nodes/ComfyUI-TTS2Whisper/requirements.txt if you see anything odd.

Where people get burned:

• replace_input_str must be the complete script, in order. The node advances through it segment by segment using character counts, so if your segment list is badly out of sync with the script (Whisper split very differently than expected), the mapping drifts and you get garbled slices. Keep the script contiguous - no missing lines.
• Run the Injector before the TimeFix and Merge nodes. They make decisions based on text length, and they should be judging your clean text, not Whisper's version.
• It doesn't touch timing. If the times are wrong, that's TimeFix's job. This node only fixes what the subtitles say.

This is a genuinely useful node for anyone doing TTS → Whisper captioning, and it's the one in the pack with actual engineering behind it - the pinyin-aware alignment is a nice touch you won't find in the generic string utils.