Render Lyric Captions

Almost every lyric video you've seen made by someone in this community was finished in CapCut, because ComfyUI had nothing to say about captions. This is the first node I've seen that keeps the whole job in the graph: song in, lyrics in, and back comes a VIDEO with the words animated over the original soundtrack, plus an SRT and a timing JSON you can hand-edit.

What it's for

People generating music locally with ACE-Step keep hitting the same wall: the track is fine, the vocals are rough, and they want it on video with words. The old workflow was render the clips in ComfyUI, escape to an editor, hand-time the text. This node removes the escape - generate or load the song, audition it with Load Audio, render captions, ship it through Save Video. It isn't a karaoke machine and it won't invent a performance. It takes audio, aligns your lyrics against what's actually sung, and burns text into frames.

How it works

Under the hood it makes one recognition pass over the full song (local Whisper large-v3 via stable-ts, the same stack as the pack's Transcribe Audio node) to find where your supplied lines are sung, then forced-aligns your words inside those matched windows. Original wording is matched in sequence, including repeated sections, and recognized text is never substituted for what you wrote.

Connect vocals - a full-song stem, time-zero aligned, duration within 50 ms - and it uses that for detection and alignment; rendering always uses the original mix, so you keep the soundtrack. On the author's own qualification runs, separation improved vocal detection without improving lyric matching, so don't treat it as a fix-all.

Two consequences follow:

• Lines it can't match are omitted from the captions, not shoved into an instrumental gap on a guess. The report lists them, and its coverage percentage is a line-coverage figure, not a timing-accuracy score.
• Nothing is interpolated. A partially timed line stays visible across its recognized span, but only words with real timings highlight.

Inputs and outputs that matter

Required and easy to miss: audio and lyrics are both full-song inputs, and lyrics is socket-only - wire a String or Multiline Text node into it, there's no widget to type into. Same for trim_start and duration: FLOAT sockets with no default, so they must be wired. duration = 0 renders the remainder of the song, and the two of them trim audio and captions together.

The rest you'll actually touch: language, device (cpu int8, cuda float16, auto picks), mode (whole-line, active-word, floating-words, floating-lines, rotating-words, rotating-lines), font_file and font_size, the colour fields, enable_glow with its five controls, and caption_transparency to fade text without deleting it. Animation controls - fade_in, fade_out, min_display, max_words, max_simultaneous, seed, circle_radius, float_distance - only appear for the modes that use them.

Optional corrected_timing is the power tool: paste reviewed timing JSON in and inference is skipped entirely.

Outputs are video, srt, report, timing_json, cleaned_lyrics. Wire video into Preview Video [Eclipse] to eyeball it, then into Save Video [Eclipse] to write the file. srt is clip-relative; timing_json is always full-song and doubles as corrected-timing input:

{"version": 1, "duration": 210.4, "language": "en",
 "lines": [{"text": "Hello world!", "start": 2.0, "end": 4.0,
   "words": [{"text": "Hello", "char_start": 0, "char_end": 5, "start": 2.0, "end": 2.8}]}]}

Character offsets count Python characters and must slice the line exactly, or the edit is rejected.

Install

Same pack, but this node has two real prerequisites: the Whisper files and fonts.

cd ComfyUI/custom_nodes
git clone https://github.com/r-vage/ComfyUI_Eclipse
python -m pip install -r ComfyUI_Eclipse/requirements.txt   # stable-ts, faster-whisper, fonttools

mkdir -p ComfyUI/models/whisper/large-v3
huggingface-cli download Systran/faster-whisper-large-v3 \
  --revision edaa852ec7e145841d8ffdb056a99866b5f0a478 \
  --local-dir ComfyUI/models/whisper/large-v3

Everything is verified against that pinned revision, and nothing downloads automatically. Fonts live in ComfyUI/models/fonts/ (TTF/OTF); if that folder is empty, Eclipse copies in its bundled fonts on first use - which is why the default is Roboto Regular unless you install something nicer.

Troubleshooting

"Missing large-v3 files" or "Unverified large-v3 artifact" means the model folder isn't the pinned conversion. Don't mix and match files from different repos.

A missing-value error you didn't cause: trim_start and duration have no widgets by design. Wire two FLOAT nodes.

Captions correct but sparse. Check the report before blaming the renderer - unmatched lines are excluded by design, and text_match_coverage tells you what fraction of lines were anchored. Sustained notes, dense production, long intros and wrong lyrics all degrade this.

Missing glyphs, or Arabic/CJK rendering badly. Pick a font that covers the script; Pillow needs RAQM for shaping and RTL, and the node errors before encoding rather than producing boxes.

Don't reach for GetVideoComponents on the result. Four minutes at 720×1280/24 fps is roughly 59 GiB of float32 frames. The renderer is file-backed and streams straight into Save Video; keep it that way.

The genuinely useful trick from the author's docs: put an Any Multi-Switch Mixed node after the renderer - captioned VIDEO in slot one, a still image in slot two - feeding Save Video. Mute the renderer and you save the still with the audio instead.