Nodes/Audio nodes/Encode encodec codebooks
ComfyUI Node

Encode encodec codebooks

Turn audio into the tokens Bark thinks in

By gitmyloΒ·Created 2 years agoΒ·Updated about a year agoΒ· 12
Encode encodec codebooks
  • encodec_model
  • audio
  • codebooks

The reverse direction: encodec_encode takes real audio and compresses it down into the discrete EnCodec codebook tokens that Bark's models work with. If you've ever wanted to hand Bark a piece of audio to reference - rather than a text prompt - this is how that audio gets into the token world.

Why you'd want audio-as-tokens

Bark only ever sees tokens. It can't listen to a waveform; it can only predict codebook arrays. So any time you want Bark to be influenced by an existing clip - the classic case being voice cloning - that clip has to be converted into the same codebook format first. That's this node's one job: AUDIO in, 8-row EnCodec codebook array out, using the 24 kHz/6 kbps codec from encodec_loader.

Inputs & outputs

  • encodec_model - from encodec_loader.
  • audio - any node that outputs ComfyUI's native AUDIO type. ComfyUI has its own audio handling (the pack's example workflow ends at PreviewAudio), and tools like Video Helper Suite are a common source when you want audio pulled out of a video file.

The output, codebooks, is what makes this node important in the voice-cloning chain: it's the fine prompt for save_speaker_npz. From there the full loop is:

  1. encodec_encode β†’ full 8-codebook array (fine prompt)
  2. bark_encodec_coarseify β†’ first 2 rows (coarse prompt)
  3. bark_hubert_encode + bark_hubert_quantize β†’ semantic tokens (semantic prompt)
  4. save_speaker_npz β†’ a .npz you can load later for consistent voices.

You can also skip the speaker-file detour entirely: encode a clip and feed the codebooks straight into bark_generate_fine's fine_history (or the coarse stage's history) for one-off style transfer.

Install & gotchas

Pack install, same as everything here - ComfyUI Manager β†’ "Audio nodes", or:

cd ComfyUI/custom_nodes
git clone https://github.com/gitmylo/ComfyUI-audio-nodes

then restart. No extra downloads beyond the bundled codec.

The gotchas are about your source audio, not the node. EnCodec encodes whatever you give it, so garbage in, garbage out: a quiet, noisy, or heavily-compressed clip will produce codebooks that reproduce all of that. Speech at 24 kHz is the intended use; feeding it 48 kHz studio audio is fine (it resamples internally), but don't expect the codec to "clean up" a bad recording. And remember the output is a lossy compression - the codebooks aren't a perfect digital copy, they're EnCodec's learned best guess, which is precisely why they're useful as style references and not as lossless audio storage.

CategoryπŸ”‰ AudioNodes/general/encodec

Inputs (2)

NameTypeDefaultDescription
encodec_modelEncodecModelβ€”
audioAUDIOβ€”

Outputs (1)

NameTypeDescription
codebooksEncodecCodeBooksβ€”