Nodes/ComfyUI_MooER/MooER_LoadModel
ComfyUI Node

MooER_LoadModel

The 15-Gig Model Gatekeeper Behind ComfyUI Speech-to-Text

By smthemex·Created 2 years ago·Updated about a year ago· 5
MooER_LoadModel
    • model
    • tokenizer
    • cmvn
    Qwen2_repoQwen/Qwen2-7B-Instruct
    Qwen2_diff
    mooer_repomtspeech/MooER-MTL-5K
    mooer_diff
    use_torch_musafalse
    use_modelscopefalse
    encoder_name
    mode_choice

    ComfyUI isn't usually the place you go to transcribe audio. But that's exactly what this pack does, and MooER_LoadModel is the unglamorous half of it: the node that drags roughly 15GB of model weights into your graph before any transcribing can happen.

    MooER (摩耳) is an LLM-based speech recognition and translation model from Moore Threads, the Chinese GPU maker. "LLM-based" isn't marketing here. The pipeline is a Paraformer audio encoder feeding a full Qwen2-7B-Instruct - the LLM itself decodes the audio embeddings into text, not a lightweight CTC head. This ComfyUI port comes from smthemex, a prolific author with two dozen packs that wrap freshly-released models into the graph fast (ParlerTTS, EchoMimic, FoleyCrafter, TwinFlow, SenseNova_U1...). Quality of those ports varies; this one is a fairly faithful translation of MooER's own inference script, quirks included.

    On a run, this node downloads both models (auto, from HuggingFace - or ModelScope if you flip use_modelscope), freezes the encoder and LLM, merges in one of three task LoRAs, and hands you the pieces the sampler needs. Downloads go into ComfyUI/models/diffusers/ - yes, diffusers, not checkpoints - which is also where its dropdowns look for already-downloaded models.

    The inputs that matter:

    • Qwen2_repo and mooer_repo - repo-id strings. Defaults are right; leave them.
    • mode_choice - the task adapter: ASR_AST (transcribe and translate in one pass), ASR, or AST.
    • use_modelscope - flip it on when HuggingFace is slow or blocked; it needs the modelscope package, which the requirements install.
    • use_torch_musa - only for Moore Threads (MUSA) GPUs. Leave it off everywhere else.
    • encoder_name - lists whisper and hubert, but only paraformer actually works today.
    • Qwen2_diff / mooer_diff - dropdowns that fill with models already in models/diffusers.

    The outputs are three MODEL-typed slots - model, tokenizer, cmvn - and all three feed MooER_Sampler's inputs. (The typing is a white lie: they're a combined encoder+LLM, a Qwen tokenizer, and CMVN stats. ComfyUI only lets them say MODEL.)

    Install via ComfyUI Manager (search "ComfyUI_MooER") or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/smthemex/ComfyUI_MooER.git
    python -m pip install -r requirements.txt
    

    The requirements file installs just modelscope and fire. The real dependencies - transformers, peft, torchaudio - are commented out because they're expected in your base ComfyUI environment. If imports go red, install those. The first run downloads Qwen2-7B (four safetensors shards, ~15GB) plus the MooER weights, so give it time and disk.

    Two traps worth knowing. First, once the models are downloaded, running again with the default repo id and "none" in the dropdown raises an error - it's a guard against silently overwriting your local copy. Pick the model from the Qwen2_diff / mooer_diff dropdown instead and it loads from disk. Second, the 80K model is ASR-only for now; ask it for translation and the code quietly substitutes 5K paths.

    If you just want a transcript, LoadAudio → MooER_LoadModel → MooER_Sampler → ShowText is the whole graph. LoadModel is the boring, heavy prerequisite - and the part where most things go wrong.

    CategoryMooER

    Inputs (8)

    NameTypeDefaultDescription
    Qwen2_repoSTRINGQwen/Qwen2-7B-Instruct
    Qwen2_diffCOMBO1 options: none
    mooer_repoSTRINGmtspeech/MooER-MTL-5K
    mooer_diffCOMBO1 options: none
    use_torch_musaBOOLEANfalse
    use_modelscopeBOOLEANfalse
    encoder_nameCOMBO4 options: paraformer, whisper, hubert, paraformer
    mode_choiceCOMBO3 options: ASR_AST, ASR, AST

    Outputs (3)

    NameTypeDescription
    modelMODEL
    tokenizerMODEL
    cmvnMODEL