ElevenLabs Pro - Forced Alignment

Speech-to-text answers "what was said." Forced alignment answers "exactly when was each word said" - and it does it by taking the transcript as a given rather than guessing at it. You hand it an audio clip and the words that are in that clip, and it hands back a timing for every one of those words.

In a ComfyUI graph that's the difference between captions that land on the syllable and captions that drift a half-second behind by the end of the paragraph.

Why you'd reach for it

If you generated the audio yourself, you already have the transcript. It's sitting in the text box you typed into the TTS node. Paying a transcription model to re-listen and re-guess the words is both wasteful and strictly worse - a transcriber can mishear a name and give you timings for a sentence that was never spoken. Alignment can't do that, because there's no transcription step: the words are input, not output.

So the two jobs it exists for are subtitles and sync. Subtitles, because words_json drops straight into the pack's Subtitle Export (SRT/VTT) node and comes out as cues. Sync, because word timings are what let you cut an edit, retime a shot, or check that an avatar's mouth work actually tracks the line it's supposed to be tracking.

It's also worth saying what it isn't: not a way to fix a bad transcript. Feed it a paraphrase and it will align the paraphrase to the audio and produce timings that look completely legitimate and are off by a word everywhere.

How it works

The node converts your AUDIO input into 16-bit PCM WAV bytes - via soundfile when it's available, with a hand-rolled WAV encoder as the fallback so you don't need an audio decoder just to upload - and POSTs it as a multipart form to ElevenLabs' /v1/forced-alignment endpoint, alongside the text field. It then writes the API's words array out as indented JSON and returns the response's loss value as a float.

The timeout on that request is 300 seconds, which tells you the honest thing about this endpoint: it's not instant, and long clips take real time. Budget for it.

Inputs and outputs

Only three inputs, and two of them are required:

• api_key - your ElevenLabs key, or leave it blank and set ELEVENLABS_API_KEY.
• audio - an AUDIO connection. Not a filename, not a video. Use Audio Trim or Audio Concat from this pack if you need to prepare a clip first.
• text - the transcript as spoken. The tooltip says "any format; speaker labels are not supported," and it means it: a script full of SPEAKER 1: prefixes will align those labels as if they were spoken words.

There are no optional inputs at all. Nothing to tune, no model dropdown, no granularity setting. Which is refreshing, honestly - there's one way to use this and the node doesn't pretend otherwise.

Outputs:

• words_json - string containing the per-word timing array. Same shape as the words_json that the pack's Speech to Text node produces, so Subtitle Export (SRT/VTT) accepts either and you can swap between them mid-project without rewriting the graph.
• loss - a float, the aligner's own error metric. Lower is better. It isn't a quality score in any absolute sense, but if you run the same clip twice with two different transcripts and one loss is dramatically worse, that's your answer about which transcript actually matched.

Install

Same pack, same two dependencies:

cd ComfyUI/custom_nodes
git clone https://github.com/IxMxAMAR/ComfyUI-ElevenLabs-Pro.git
pip install -r ComfyUI-ElevenLabs-Pro/requirements.txt

Restart ComfyUI - the node shows up under ElevenLabs Pro/Audio. Or just search "ElevenLabs Pro" in ComfyUI Manager and let it do the clone. requests and soundfile are the entire dependency list; there are no weights to download and nothing that installs into the same torch/transformers stack your video nodes are fighting over, which is a genuine relief in an ecosystem where audio packs are usually the thing that breaks your environment.

Common issues

"Text input is empty." Raised before the API call - nothing was billed, you just left the transcript box blank.

Timings are consistently off by a second or two. Nearly always the transcript, not the audio. Extra intro music, a leading "um," a cut verse of the script still in the box - every mismatch shifts everything after it. loss will usually show it.

Speaker labels in the transcript. They're not supported and they consume words from the alignment. Strip the labels, keep one continuous transcript.

No node output, no error, suspiciously fast queue. It's an API node, not an output node - ComfyUI walks backward from output nodes. If nothing downstream consumes words_json, the branch never executes. Wire it into Subtitle Export (SRT/VTT) and save that text somewhere.

Billing surprises. Every paid node in the pack caches by input hash: same audio, same text, re-queue is free; change the transcript and it's a new call. Note the audio hash is shape-and-sample-rate based, so a different clip of identical length and sample rate can hit the cache - if you retime an edit at the same duration, sanity-check the output.