ComfyUI Node Runs on cloud

Load VQVAE

Load the motion tokenizer that drives MTVCrafter animation

By kijai·Created about a year ago·Updated 2 months ago· 6,651
Load VQVAE
    • vqvae
    model_name

    If you landed here you're almost certainly building an MTVCrafter workflow, because that's the one thing this node exists for. It loads the VQ-VAE that MTVCrafter uses to turn human poses into motion tokens. On its own it does nothing visible - it's the first link in a three-node chain, and without it the other two won't run.

    Quick orientation, because "VQVAE" out of context is confusing. This is not the VAE that encodes and decodes your video pixels. MTVCrafter ("4D Motion Tokenization for Open-World Human Image Animation") represents motion - the way a body moves through space over time - as a sequence of discrete tokens, the same way a language model represents text. The VQ-VAE is the codebook that does that tokenizing: it takes detected 4D poses and quantizes them into motion tokens the Wan model can be conditioned on. This node just gets that codebook into memory.

    How it fits the chain

    Three nodes, in order:

    1. LoadVQVAE (this one) - loads the motion tokenizer.
    2. MTVCrafterEncodePoses - feeds detected poses through that tokenizer to produce a motion object.
    3. WanVideoAddMTVMotion - injects that motion into your Wan image embeds so the generation follows it.

    So the output of this node is a dependency, not a result. You wire it forward and forget about it.

    The input and output

    There's exactly one input: model_name, a dropdown that lists the VQ-VAE files it finds in your models folder. In the brief its choices come up empty, which is the tell that no weights are installed yet - the dropdown populates from files on disk, so an empty list means you haven't downloaded the MTVCrafter VQ-VAE. Grab it (Kijai mirrors the WanVideoWrapper model files on HuggingFace) and drop it where the wrapper expects it, and the name shows up.

    The single output is vqvae (VQVAE type), which plugs straight into MTVCrafterEncodePoses and nothing else.

    How to install it

    The node comes with Kijai's WanVideoWrapper. ComfyUI Manager is the low-friction route: Custom Nodes Manager, search ComfyUI-WanVideoWrapper, install, restart. By hand:

    cd ComfyUI/custom_nodes
    git clone https://github.com/kijai/ComfyUI-WanVideoWrapper
    pip install -r ComfyUI-WanVideoWrapper/requirements.txt
    

    then restart. The pack code is small; the weight is in the models. For MTVCrafter you need the VQ-VAE file this node loads, plus a pose detector upstream and a full Wan setup downstream. It's not a casual "install and click generate" node - it's part of a research-grade animation pipeline, and MTVCrafter itself is a niche technique with only a handful of community threads, so expect to do some assembly.

    Common issues & troubleshooting

    The dropdown is empty. This is the number-one problem and it's not a bug - the node lists files it can see, and you haven't put the VQ-VAE file in place. Download it and restart ComfyUI so the folder gets rescanned.

    You confused it with your video VAE. Don't wire your normal Wan VAE (WANVAE) into anything expecting a VQVAE, or vice versa - they're different types for different jobs. The VQVAE tokenizes motion; the WANVAE decodes pixels.

    It loads but nothing animates. This node only loads. If the final video ignores your motion, the problem is downstream - check that MTVCrafterEncodePoses actually got valid poses and that WanVideoAddMTVMotion's strength isn't zeroed.

    CategoryWanVideoWrapper

    Inputs (1)

    NameTypeDefaultDescription
    model_nameCOMBOThese models are loaded from 'ComfyUI/models/vae'

    Outputs (1)

    NameTypeDescription
    vqvaeVQVAE