Nodes/msch-comfyui-nodes/πŸŸͺ Lyric Sync β€” Word Mosaic
ComfyUI Node

πŸŸͺ Lyric Sync β€” Word Mosaic

Turn your music video into a living word grid β€” and keep the singer visible underneath

By mariobillyΒ·Created a day agoΒ·Updated a day agoΒ· 0
πŸŸͺ Lyric Sync β€” Word Mosaic
  • images
  • timing
  • palette_in
  • images
β—„frame_rate25.000β–Ί
β—„cols56β–Ί
β—„font_pathC:\Windows\Fonts\tahoma.ttfβ–Ί
β—„letter_scale0.72β–Ί
β—„scroll_dirdownβ–Ί
β—„scroll_speed0.12β–Ί
β—„color_density0.13β–Ί
β—„flicker_period4β–Ί
β—„wordbox_density0.60β–Ί
β—„wordbox_scale2.3β–Ί
β—„bg_keytrueβ–Ί
β—„bg_tol0.00β–Ί
β—„threshold_bias0.00β–Ί
β—„invert_subjectfalseβ–Ί
β—„fg_detail3.0β–Ί
β—„posterize7β–Ί
β—„subject_max0.95β–Ί
β—„subject_gamma0.60β–Ί
β—„fg_gap0β–Ί
β—„text_color#15151Eβ–Ί
β—„seed7β–Ί
β—„palette#FFFFFF,#FFFFFF,#7DEFA1,#5FE39A,#F25CC1,#FF74B8,#63D6F0,#9B5DE5,#B388F0,#F5B8D6β–Ί

If LyricSyncOverlay is the tasteful karaoke captions, LyricSyncMosaic (display "πŸŸͺ Lyric Sync - Word Mosaic") is the show-off sibling. It rebuilds your entire video frame as a scrolling grid of colored letter cells - think a giant text marquee - and then does the clever part: it detects the subject in each frame and keeps the current lyric's words glowing on top while the singer stays recognizable through the grid. It's the "lyrics as art direction" node, not the "lyrics as subtitles" node, and it's genuinely striking when it works.

How it works

Every frame is converted into a grid of cols letter cells (default 56 across - higher is finer and denser). The grid scrolls (scroll_dir and scroll_speed, default: drifting down at 0.12 cells per frame), and a fraction of cells light up with palette colors (color_density, flicker_period) to give it that animated LED-marquee shimmer. Meanwhile the lyrics are read from the same LYRIC_TIMING that drives the Overlay node - supplied by the pack's Lyric Sync - Align node - and the current line's words are drawn as highlighted word boxes whose size you control with wordbox_scale and density with wordbox_density.

The genuinely clever input is bg_key. The song's video frames usually have the singer or subject against some backdrop; bg_key keys out the background so the grid can sit under the subject, with bg_tol (0 = auto) controlling how much of the frame counts as background. Tonal controls - threshold_bias, invert_subject, posterize, subject_gamma - then shape how the subject reads through the text. It's a lightweight background-removal path tuned for the "person on a plain backdrop" case, and it's honest about its limits: best on plain backgrounds, as the tooltip says.

The inputs that matter

Beyond images + timing (LYRIC_TIMING from Align), the ones beginners actually set:

  • frame_rate - must match your video's fps, or word boxes drift off the vocals.
  • cols - the density personality switch. Low (20–30) = chunky, readable marquee; 56 default is fine; higher gets fine-grained and busy.
  • palette - comma-separated hex colors for the lit cells. The optional 🎨 Lyric Sync - Palette input overrides this text field and is the cleaner way to tune colors without editing a comma string.
  • bg_key / bg_tol - turn keying on and set tolerance; this is what makes the subject visible instead of a wall of letters.

scroll_speed, color_density, and wordbox_density control the liveliness - think of the defaults as "calm and legible" and raise them for energy.

Wiring it

LyricSyncAlign β†’ LyricSyncMosaic β†’ Save Video. Align needs your audio and pasted lyrics to produce the timing; Mosaic consumes it plus your frame batch and outputs images. Like the Overlay node, it returns frames only - connect audio separately to the encoder.

Install

Part of the unified MSCH Nodes pack (search "MSCH Nodes" in ComfyUI Manager, or msch-comfyui-nodes):

cd ComfyUI/custom_nodes
git clone https://github.com/mariobilly/msch-comfyui-nodes.git
cd msch-comfyui-nodes
python -m pip install -r requirements.txt

Restart ComfyUI afterward. No models download at startup. Note font_path defaults to a Windows font (tahoma.ttf); on Linux/macOS set it to a local .ttf or rendering fails.

Gotchas

This node has the longest control surface of the two lyric renderers, and the failure modes track the knobs. The subject disappears β†’ your bg_key/bg_tol are off and the grid paints over the singer; dial bg_tol down toward keeping more of the frame as subject. Words drift β†’ frame_rate mismatch, same as Overlay. And if the whole thing looks like static, your source video's background is probably too busy for keying to find the subject - the tooltip is not kidding that this is "best for plain backgrounds." Start there, get the singer reading through the grid, and only then start raising density and scroll speed.

CategoryLyricSync

Inputs (25)

NameTypeDefaultDescription
imagesIMAGEβ€”
timingLYRIC_TIMINGβ€”
frame_rateFLOAT25.0001–240β€”
colsINT5610–220Letter cells across the frame (higher = finer/denser).
font_pathSTRINGC:\Windows\Fonts\tahoma.ttfβ€”
letter_scaleFLOAT0.720.3–1β€”
scroll_dirCOMBOdown5 options: down, up, left, right, none
scroll_speedFLOAT0.120–1Cells per frame the grid drifts.
color_densityFLOAT0.130–0.6Fraction of letter cells that light up with colour.
flicker_periodINT41–60Frames between colour-cell flicker changes (lower = faster).
wordbox_densityFLOAT0.600–1How many of the current line's word boxes show at once.
wordbox_scaleFLOAT2.31–5Word-box height relative to a letter cell.
bg_keyBOOLEANtrueKey out the backdrop to find the subject (best for plain backgrounds).
bg_tolFLOAT0.000–10 = auto. Lower keeps more of the frame as subject.
threshold_biasFLOAT0.00-0.4–0.4β€”
invert_subjectBOOLEANfalseβ€”
fg_detailFLOAT3.01–6β€”
posterizeINT72–32β€”
subject_maxFLOAT0.950.2–1β€”
subject_gammaFLOAT0.600.3–3<1 brightens a dark subject; >1 darkens.
fg_gapINT00–4β€”
text_colorSTRING#15151Eβ€”
seedINT70–99999β€”
palette_inoptPALETTEConnect a 🎨 Lyric Sync β€” Palette node (overrides the text below).
paletteoptSTRING#FFFFFF,#FFFFFF,#7DEFA1,#5FE39A,#F25CC1,#FF74B8,#63D6F0,#9B5DE5,#B388F0,#F5B8D6Comma-separated hex colours (used if no Palette node connected).

Outputs (1)

NameTypeDescription
imagesIMAGEβ€”