Prompt To Semantic

Prompt To Semantic is the text-understanding half of Mixlab's Fish Speech pipeline, and once you see the shape of its inputs, the whole thing clicks into place as a voice-cloning setup rather than a generic TTS node.

Fish Speech (rebranded OpenAudio, and a real, actively-discussed open-source project rather than something Mixlab built from scratch) generates speech in two stages. This node handles the first: turning text into a sequence of discrete "semantic" audio tokens, conditioned on a reference voice. Load VQGAN, elsewhere in this same FishSpeech category, handles the second stage - decoding those tokens back into an actual waveform. This node's codes output (typed NUMPY) is exactly what feeds into that decode step.

The voice-cloning setup shows up directly in the required inputs: text is what you want spoken, but prompt_text and prompt_tokens are the reference - the transcript of a sample recording in the target voice, plus that sample's own semantic tokens (extracted by a separate upstream node this pack provides for that purpose, not this one). Give it a clean reference clip and transcript, and the model attempts to continue speaking your new text in that same voice.

Everything else on this node is a standard LLM-style sampling knob, because Fish Speech's text-to-semantic step is transformer-based under the hood: top_p (0.6–0.9, default 0.7) and temperature (0.6–0.9, default 0.7) control how much variation creeps into the token sequence, repetition_penalty (1.0–1.5, default 1.2) discourages the model from looping on the same sound, and seed (default 42) is there for reproducibility - though see the caveat below before you trust it too far. max_new_tokens caps how long the generated sequence can run, chunk_length breaks long text into manageable pieces, iterative_prompt (yes/no) controls whether earlier generated chunks feed back in as additional context for later ones, and compile (yes/no) triggers torch.compile for faster repeated generations - at the cost of a noticeably slower first run while it compiles. precision (bf16 or half) and device (cuda or cpu) round out the performance settings.

Set expectations honestly before you invest time here: community reports on Fish Speech's voice cloning are mixed rather than glowing. People describe results as "not as good as it should be, not robust as well" compared to some competing options, and at least one recent report specifically found that locking the seed didn't produce repeatable output between runs on the same input - so don't be surprised, or assume you've misconfigured something, if two runs with identical settings sound noticeably different. It's a legitimate open project worth experimenting with, just don't expect studio-consistent output on the first try.

Install is the standard pack-wide route:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
cd comfyui-mixlab-nodes
install.bat

or via ComfyUI Manager (search "comfyui-mixlab-nodes"), or pip3 install -r requirements.txt in a venv. One honest gap to flag: the pack's README, thorough almost everywhere else about model download locations, says nothing about where the Fish Speech checkpoints for this node actually go - you'll need to check the upstream fishaudio/fish-speech project or the pack's own GitHub discussions for that detail before this node will run.