🎤 Speech Recognition

This node listens to an audio file and spits out a transcript - with timing - so Mana Nodes can auto-generate captions synced to speech. It's the front half of the "drop in an audio track, get animated subtitles" workflow. Feed its output into a Text to Image Generator and the words appear on-screen roughly when they're spoken.

Why you'd reach for it

The whole reason to auto-caption inside ComfyUI is to avoid transcribing by hand and eyeballing timings. This node does the transcription and the timing in one shot, and its output is shaped specifically to drive Mana's text renderer. If you're building talking-head clips, lyric videos, or captioned shorts entirely in a Comfy graph, this is where the words come from.

How it works

Under the hood it runs a Wav2Vec2 model from HuggingFace - Meta's self-supervised speech-recognition architecture. That's a deliberate design choice, and an honest word about it: Wav2Vec2 is solid but it's not the current darling of the ASR world (a lot of caption workflows now reach for Whisper). Wav2Vec2 tends to output ALL CAPS with no punctuation, which is actually fine for punchy kinetic captions but noticeable if you wanted clean prose. The pack even ships a spell-check pass to clean up the raw output.

The inputs that matter

• audio_file - a path or URL to the audio. In a full workflow this usually comes from a Split Video node that extracted the audio track for you.
• wav2vec2_model - which model to use. The dropdown lists a thousand Wav2Vec2 checkpoints from HuggingFace; the default jonatasgrosman/wav2vec2-large-xlsr-53-english is a strong general English pick. For other languages, this is the knob you change - there are checkpoints for Italian, Japanese, Bengali, and plenty more.
• spell_check_language - the language for the cleanup pass (English, Spanish, French, and eight others). Match it to your audio.
• framestamps_max_chars - default 25. How many characters accumulate on screen before the caption line clears and starts building again. This directly controls how much text is visible at once.
• fps - default 30. Must match your video's frame rate, because the timing is converted to frame numbers. Get this wrong and captions drift.
• uppercase - default on; whether to force caps.

Outputs: transcription (the special TRANSCRIPTION type - feed this into the generator's transcription input), plus three string forms - raw_string (the plain sentence), framestamps_string (text keyed to frame numbers), and timestamps_string (every word with start/end times as JSON).

Installing it

Comes with Mana Nodes. ComfyUI Manager → search Mana Nodes → install → restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/ForeignGods/ComfyUI-Mana-Nodes.git

Install requirements (pip install -r requirements.txt; the python_embed variant on Windows portable) and restart. Heads-up on the first run: the chosen Wav2Vec2 model downloads from HuggingFace on first use - the default large model is over a gigabyte - so the initial run stalls while it fetches. That's expected, not a hang.

Common issues

The number-one caption problem is fps mismatch. The node bakes timing into frame numbers, so if the node's fps doesn't equal your actual video fps, the words land early or late and the drift compounds over the clip. Set them equal.

Second: accuracy. Wav2Vec2 mishears words, especially on noisy or music-heavy audio, and the caps/no-punctuation output isn't to everyone's taste. The intended fix is built into the workflow - save framestamps_string with Save/Preview Text, correct the mistakes by hand, then paste the corrected framestamps straight into the generator's text field and disconnect the transcription input. Third, for non-English audio, remember to change both the wav2vec2_model (to a matching-language checkpoint) and spell_check_language.