Nodes/Audio nodes/Load HuBERT quantizer model (bark)
ComfyUI Node

Load HuBERT quantizer model (bark)

The dictionary that maps voice to tokens

By gitmylo·Created 2 years ago·Updated about a year ago· 12
Load HuBERT quantizer model (bark)
    • model
    ◄model▾►

    The voice-cloning chain has a missing link: HuBERT gives you continuous vectors, but Bark thinks in discrete tokens - 10,000 of them. load_bark_hubert_quantizer loads the small model that bridges that gap: gitmylo's custom tokenizer, an LSTM that maps HuBERT's 768-dimension vectors onto Bark's semantic vocabulary. It's the translation dictionary between "how this voice sounds" and "what token number Bark uses for it."

    The three models you can pick

    The node is one dropdown, model, with three choices:

    • quantifier_hubert_base_ls960.pth - the base quantizer, trained on HuBERT layer-9 features (the README's recommended pairing).
    • quantifier_hubert_base_ls960_14.pth - a variant trained on layer-14 features (the _14 suffix is the giveaway). Use it if you're extracting HuBERT vectors at layer 14.
    • quantifier_V1_hubert_base_ls960_23.pth - a newer V1 revision, also layer 23. Try this one if base quality isn't cutting it; it's a training-quality revision rather than a different architecture.

    The matching matters: quantizer and HuBERT layer should agree. Feed layer-9 vectors into the layer-14 quantizer and you're translating from a language the dictionary doesn't know - the tokens come out but the voice fidelity suffers. The one that ships as the default in the pack's code is quantifier_hubert_base_ls960_14.pth, so if you keep HuBERT at layer 9 (the default in bark_hubert_encode) you'll want the base _ls960.pth instead - a classic easy-to-miss mismatch.

    Mechanism & output

    On load it downloads the chosen .pth from the GitMylo/bark-voice-cloning Hugging Face repo into ComfyUI/models/bark/quantizers/ (first run only), then wraps it in a Comfy model patcher. Like the HuBERT loader, it's hardcoded to CPU - fine for a one-shot tokenization pass. The output is a single model that feeds bark_hubert_quantize.

    Install & notes

    Standard pack install:

    cd ComfyUI/custom_nodes
    git clone https://github.com/gitmylo/ComfyUI-audio-nodes
    

    (or ComfyUI Manager → "Audio nodes"), restart, and let the first load download the quantizer.

    If cloning results sound off, this dropdown is a real knob worth turning - different quantizer revisions genuinely produce different character. But it's also the place to manage expectations: the quantizer is trained on HuBERT base, which is English-heavy LibriSpeech territory. It'll tokenize other languages, but accent and language fidelity degrade fast outside what it learned.

    Category🔉 AudioNodes/bark/cloning

    Inputs (1)

    NameTypeDefaultDescription
    modelCOMBO3 options: quantifier_hubert_base_ls960.pth, quantifier_hubert_base_ls960_14.pth, quantifier_V1_hubert_base_ls960_23.pth

    Outputs (1)

    NameTypeDescription
    modelBarkHuBERTQuantizerModel—