Audio To Prompt

First thing to clear up, because the name is misleading: this node does not turn audio into a text prompt for image generation. It's a piece of the Fish Speech text-to-speech pipeline, and the "prompt" here means the audio tokens a voice-cloning TTS model uses as its reference - the "prompt" you give a voice model so it speaks in a target voice. If you came here hoping to caption a sound clip in words, this isn't that node.

What it actually does

Fish Speech (and voice-cloning TTS in general) works in two stages. A neural audio codec - a VQGAN - compresses a waveform into a compact sequence of discrete tokens, and the TTS model generates in that token space, then decodes back to sound. To clone a voice, you first encode a reference recording of that voice into codec tokens. That encoded reference is what conditions the model.

AudioToPrompt is that encode step. It takes a VQGAN codec model and an audio clip, runs the audio through the codec, and hands back the token representation plus a reconstructed version of the audio. Those tokens are the "prompt" that downstream Fish Speech nodes use to make new speech sound like your reference.

The inputs and outputs

• vqgan - the loaded VQGAN codec model (a VQGAN type, produced by a separate loader node in the Fish Speech chain). Required - this is the codec doing the encoding.
• audio - the voice clip to encode, an AUDIO object. This is your reference sample.
• device - cuda or cpu. Use cuda if you have the GPU for it; cpu works but is slower.

Outputs: restored_audio (an AUDIO - the clip decoded back from tokens, so you can hear how faithfully the codec captured it) and prompt_tokens (a NUMPY array - the actual encoded tokens that feed the TTS step).

Installing it

Get the pack via ComfyUI Manager (search mixlab, install comfyui-mixlab-nodes, restart), or clone it:

cd ComfyUI/custom_nodes
git clone https://github.com/shadowcz007/comfyui-mixlab-nodes

then install requirements and restart. This node is part of the pack's Fish Speech support, which means you also need the VQGAN / Fish Speech model weights loaded via the companion loader node - the codec doesn't come with the pack itself. It won't do anything without a VQGAN wired into it.

Common snags

• "Where's my text prompt?" There isn't one - the output is audio tokens, not words. This is a voice-cloning building block, not an audio captioner. If you want a spoken clip transcribed to text, you want a speech-recognition node (the pack has SenseVoice / SpeechRecognition for that), not this.
• The vqgan input is empty / node won't run. You need the Fish Speech VQGAN loader upstream, with its weights downloaded. This node is one link in a chain, not standalone.
• Reconstructed audio sounds rough. restored_audio is a codec round-trip, so some loss is expected - it's a diagnostic to confirm the encode worked, not a hi-fi output. What matters downstream is the token array, not this preview.
• CPU is crawling. Switch device to cuda if you can; codec encoding is much happier on a GPU.