SemBridge Speech + Singing

Here's a thing no pipeline of separate tools does well. You want a character to say a line, sing a phrase, then say another line - in the same voice, with the same recording character, no seam where one model hands off to another. The usual approach is to generate speech with a TTS model and singing with a singing model and cross your fingers that the two voices resemble each other. They won't.

SemBridge Speech + Singing is the node that exists because that's all one model underneath. Speech, singing and mixed speech-plus-singing are the same autoregressive generation with different task tags, so mixing them is native rather than bolted on. This node feeds the model an ordered list of sections and gets back one continuous render.

Writing segments

The segments box takes sections, in order, tagged [speech] and [singing]:

[speech]
Once upon a time, a singer walked into town.
[singing]
la 60 0.5
la 62 0.5
la 64 1.0
[speech]
And the crowd went quiet.

Singing sections use the same note syntax as the Singing node - lyric pitch seconds, optional note_type, <SP> with pitch 0 for a rest. Speech sections are plain text. The parser also accepts [tts] and [svs] as aliases and doesn't care about case, which is handy if your hands type the wrong one. A comment line starting with # is ignored.

The official JSON segment list works too - a list of {task, text} and {task, notes} objects - and that's the sane path if you're generating the structure from code. Up to 64 segments, which is more than enough given how short the audio ends up being.

One parsing detail that bites: the node decides which mode you're in by looking at your first line. If it doesn't start with a section header, it assumes JSON and tries to parse the whole box that way, then fails with a message about neither section syntax nor valid JSON. So lead with [speech], always.

Speech text also gets its whitespace collapsed - newlines inside a speech section become single spaces. Don't format a speech section like a poem and expect pauses.

One pass, one voice, one budget

The mechanism is the reason to use the node: everything you list is generated in a single continuous request. There's no stitch point, no second model, no drift - the voice that says the first line is the voice that sings and the voice that finishes. Which also means the section order defines the phrasing arc of the entire clip, and a speech section followed by singing doesn't get any special transition; the model just keeps going.

It also means the length budget is shared. max_new_patches is 500 by default - 40 seconds total for the whole segment list, not per section. A three-part example eats that fast given that singing runs at roughly 2.3 to 2.8 seconds of compute per second of audio. If a render cuts off mid-final-word, check the report's patches_generated against your cap before you blame the text.

decoder on auto picks BigVGAN-Large for mixed generation, which is the official default and the right call here. flow_steps defaults to 20 for this node, matching the official mixed examples - that's twice what speech uses and it's part of why mixed renders are slower. cfg_value 2.0, temperature 1.0, both model defaults.

The optional prompt_audio / prompt_text pair conditions the entire thing on one reference voice, same rule as the other two generation nodes: audio and its exact transcript together, or it raises. For a mixed clip, the reference decides what the whole performance sounds like, so a speech reference drags the singing toward speechy phrasing and vice versa. prompt_task lets you declare which it is instead of leaving it on auto.

Outputs

audio is mono 44.1 kHz, clamped to [-1, 1]. report is a JSON string - task will read unified, plus the decoder, patch count, stopped_early, audio_seconds, generation_seconds, realtime_factor and peak VRAM. Because mixed runs sit around 2.9 to 3.9 GiB, that VRAM number is worth a glance if you're also holding a video model.

Install

Nothing special - it's the same pack and the same weights as the other three generation nodes.

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-SemBridge
python -m pip install -r ComfyUI-SemBridge/requirements.txt
python ComfyUI-SemBridge/tools/download_model.py --models-dir ComfyUI/models/sembridge

requirements.txt is soundfile only; don't pip install sembridge (torch pin below 2.9, code already vendored). You need the V3 node API - ComfyUI v0.38.0 or newer - and there's a ready graph at workflows/gui/unified_basic.json.

Where people get burned

Empty or invalid scores inside a section. The whole request fails at parse time, with a line number pointing into the section. Build the singing sections in Score Preview first, then paste them in - it uses the same parser and no GPU.

High notes still fail. The same weakness that affects the Singing node applies here: wide leaps up in MIDI 72–79 were off by 2.4 to 4.2 semitones in testing, while MIDI 48–72 held within about half a semitone. A mixed clip that sounds fine when spoken and drunk when sung is usually a register problem, not a settings problem.

Clipping. Output is clamped, never scaled, and singing clips more readily than speech. Five of 18 automatic singing requests touched full scale.