MTV Crafter Encode Poses

This is the middle node of the MTVCrafter chain, and it's where the actual encoding happens. It takes the poses you detected from a driving video and runs them through the VQ-VAE codebook to produce a compact motion representation - the thing that later gets injected into a Wan generation to make your character move the way the driving performer did.

MTVCrafter's whole pitch ("4D Motion Tokenization for Open-World Human Image Animation") is that it doesn't drive Wan with a rendered stick-figure video the way most pose-transfer setups do. It tokenizes the 4D motion directly. That matters because the community has been circling the same insight from a few directions: the KB notes SCAIL-2's big win was dropping the stick-figure intermediate entirely, since "the things that get lost in that conversion, like depth and contact, are exactly the things that matter." MTVCrafter is another swing at that idea - keep the motion in a richer form than a flat pose image before handing it to the model.

How it works

You give it two things: the VQ-VAE from LoadVQVAE, and a set of predicted poses. It encodes those poses into motion tokens using the codebook, and outputs a motion object plus the pose data itself (handy if you want to visualize or reuse the detection). Downstream, WanVideoAddMTVMotion takes that motion object and folds it into your Wan image embeds so sampling follows the movement.

The inputs and outputs

Both inputs are required:

• vqvae (VQVAE) - the tokenizer from LoadVQVAE. No codebook, no encoding.
• poses (NLFPRED) - the detected poses. NLFPRED is the "neural localizer field" style pose prediction MTVCrafter expects; it comes from the pose-detection node upstream in the MTVCrafter workflow, not from a plain DWPose image.

Two outputs:

• mtvcrafter_motion (MTVCRAFTERMOTION) - the encoded motion. This is the payload; wire it into WanVideoAddMTVMotion.
• pose_results (NLFPRED) - the pose data passed back out, for chaining or inspection.

There's nothing here a beginner "tunes" - it's a plumbing node. The knobs that shape the final look (strength, timing) live on WanVideoAddMTVMotion, not here.

How to install it

It's part of Kijai's WanVideoWrapper. ComfyUI Manager: search ComfyUI-WanVideoWrapper, install, restart. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-WanVideoWrapper
pip install -r ComfyUI-WanVideoWrapper/requirements.txt

then restart. This node needs the MTVCrafter VQ-VAE file (loaded by LoadVQVAE) and a pose predictor that outputs NLFPRED, so the real install work is gathering the MTVCrafter model assets, not the node itself. Fair warning: MTVCrafter is genuinely niche - a handful of threads across the whole corpus - so you're on the frontier here and workflows are thinner on the ground than for VACE or Wan Animate.

Common issues & troubleshooting

Type mismatch on poses. The poses input wants NLFPRED, which is a specific pose-prediction format, not the OpenPose/DWPose IMAGE you'd feed a normal ControlNet. If ComfyUI refuses the connection, you're trying to plug a rendered pose image where a pose prediction belongs - you need MTVCrafter's own detector in front.

Empty motion downstream. If the encode runs but the character doesn't move, confirm the pose detection actually found a body. Garbage or empty poses in, empty motion out.

VQ-VAE not loaded. If LoadVQVAE's dropdown was empty because the file wasn't downloaded, this node has nothing to encode with. Fix the load first.