Semantic To Audio

Semantic To Audio is the closing node in Mixlab's Fish Speech pipeline - the one that finally turns a stream of numbers back into something you can listen to. Fish Speech (now rebranded OpenAudio, a real and actively-discussed open voice-cloning TTS project) generates speech in two stages: a text-to-semantic step converts your text, conditioned on a reference voice, into a sequence of discrete audio tokens, and a vocoder decodes those tokens back into a waveform. Mixlab's Prompt To Semantic node handles the first half. This node is the second half.

The inputs line up exactly with that job. vqgan is the loaded vocoder object - Mixlab's Load VQGAN node produces it, wrapping the "Firefly" GAN architecture Fish Speech uses for this stage. codes is a NUMPY array of the actual semantic tokens, and it's meant to come straight from Prompt To Semantic's codes output - that's the handoff between the two nodes, one produces the token sequence, this one decodes it. device picks cuda or cpu; leave it on cuda unless you're doing a quick sanity check on a machine with no GPU to spare. The single output is generated_audio, typed AUDIO, ready to preview with an audio-preview node, save to a file, or mux onto a video with something like Mixlab's own CombineAudioVideo.

There's no creative dial here - no pitch, no speed, no style knob. This node is pure decode: token sequence in, waveform out. Whatever character the speech has - pacing, prosody, how closely it matches the reference voice you conditioned Prompt To Semantic on - was already baked into codes by the time it reaches here. If the output sounds wrong, the fix almost never lives in this node.

Set your expectations on quality honestly. Fish Speech's voice cloning gets a mixed reception in the community rather than a glowing one - people report results that are "not as good as it should be, not robust as well" next to some competing TTS options, and there are specific reports of the same seed and inputs not producing repeatable output between runs. It's a legitimate, worth-trying open project, not a broken one - just go in without assuming studio-consistent results on the first pass.

Install is the pack-wide standard:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
cd comfyui-mixlab-nodes
install.bat

or via ComfyUI Manager (search "comfyui-mixlab-nodes"), or pip3 install -r requirements.txt in a venv. One honest gap: the README, thorough almost everywhere else it lists a model download, never spells out where the Fish Speech VQGAN checkpoint goes. You'll need Load VQGAN's model dropdown populated before this node has anything to decode - and getting a checkpoint into the right folder means checking the upstream fishaudio/fish-speech project or the pack's own GitHub issues, not this README.

The most common failure isn't in this node at all: an empty or missing vqgan because Load VQGAN's model dropdown had nothing in it, which throws well before Semantic To Audio ever runs. Second most common: a codes array that came from somewhere other than Prompt To Semantic, or from a Prompt To Semantic run against a different reference - the vocoder decodes faithfully, so if the semantic tokens were already off, wrong, or generated against a mismatched voice reference, the audio that comes out will be too, and no setting on this node can fix tokens that were wrong going in. If the result plays but sounds garbled or noisy rather than just "not quite the target voice," double check device matches the hardware vqgan was actually loaded onto - a CPU/GPU mismatch between Load VQGAN and this node is worth ruling out before assuming the model itself is at fault.