ProxyPose Model Loader (Bruxos)

This is the node that makes the whole ProxyPose trick possible, and it's the reason this pack isn't a one-click install. ProxyPose takes a single pixel you click in a video's first frame and returns a full 6-DoF pose trajectory (rotation + translation, frame by frame) for whatever object sits under that pixel. To do that it runs a 14-billion-parameter video model behind the scenes - specifically Wan 2.1 T2V-14B plus a LoRA - and the Model Loader is what gets that beast into memory and pointed at the pipeline.

If you've run Wan in ComfyUI before, here's the twist that makes this node necessary: ProxyPose does not use Wan's standard image-to-video conditioning. It re-writes the DiT forward (token concatenation, per-token timestep modulation, a modified RoPE) and runs its own DDIM loop with partial noise on the first frame. All of that lives in a patched forward, not in the LoRA weights, so slapping the LoRA onto a normal Wan loader won't reproduce the result. You need this node to load the model the way the ProxyPose repo expects - via its diffsynth pipeline. That's the "why a wrapper, not a native port" story, and it's the honest one.

What it actually loads

The heavy lifting is WanVideoPipeline.from_pretrained from DiffSynth, loading the Wan2.1-T2V-14B DiT, the VAE, and the T5 (umt5-xxl) text encoder. Then it applies the ProxyPose LoRA and patches the DiT. The T5 is used exactly once to encode the fixed prompt and then discarded - which is why the ~30 GB first-run download doesn't all sit on your GPU at once.

The inputs that matter, for a beginner:

• repo_path - where your cloned ProxyPose repo lives. auto (the default) uses the deps/proxypose folder the bundled installer clones into. Whatever you put here must match the Sampler node's repo_path, or nothing works.
• precision - fp8_e4m3 is the default and the right call for most people: DiT weights stored in fp8, computed in bf16. That roughly halves DiT VRAM (28 GB → ~14 GB) with a small quality cost and zero speed gain. bf16 is full precision if you have the VRAM to spare. On CPU, fp8 silently falls back to bf16 anyway.
• vram_limit_gb - cap for per-layer offload. 0 means "fit it all in VRAM"; something like 20 on a 24 GB card hands the rest to offloading.
• offload_text_encoder - leave it on. It keeps the T5 on CPU, and since the T5 runs once and is thrown away, there's no reason not to.

Outputs are pipe and gen_config - both wire into the ProxyPose Sampler. Nothing else consumes them.

Installing it for real

The README is upfront that this is a three-part install, not a Manager click:

git clone https://github.com/ruihangzhang97/proxypose
cd proxypose && pip install -e .        # pulls diffsynth, decord, etc.

cd ComfyUI/custom_nodes
git clone https://github.com/NyckM/ComfyUI-BruxosDoVFX-ProxyPose

Plus PyTorch3D, installed into the same Python your ComfyUI uses - if it lands in the wrong venv, import inference fails and you'll be staring at an import error for an hour. First run then downloads ~30 GB (Wan base) + ~600 MB (LoRA) from HuggingFace.

The part worth calling out: the pack's requirements.txt is essentially empty on purpose, because everything comes from ProxyPose's own install - and ProxyPose's pyproject.toml pins nothing and even drags in gradio you'll never use. A raw pip install -e . can cascade into a newer scipy → numpy 2.x, breaking your existing ComfyUI environment. The author knows this and ships a safe-install kit instead: double-click instalar-proxypose.bat on Windows (it finds python_embeded, clones ProxyPose into deps\proxypose, installs with --no-deps, and refuses to touch eight "untouchable" packages - torch 2.8.0+cu128, torchvision, numpy 1.26.4, scipy 1.13.1, sageattention, onnxruntime-gpu, and friends). Linux gets tools/check_pilares.py plus instalar-proxypose-seguro.sh doing the same with a dry-run safety net. If you have a working ComfyUI you care about, use those scripts, not the raw pip command.

Where people get burned

• PyTorch3D on Windows - there's no official wheel, and compiling fails on new MSVC + CUDA 12.9 (vector_types.h: expected a ">"). The installer pulls a prebuilt wheel from the MiroPsota index instead.
• You need cu130 warnings - ignore them forever. The CUDA that matters is the cu128 build inside your torch.
• fp8 + LoRA producing bad or NaN output - switch to precision=bf16 with a positive vram_limit_gb. Slower, no quality loss, and it dodges the fp8-plus-patched-forward edge case that's genuinely worth testing first.
• A diffsynth "shadow" collision - other custom nodes (comfy-dvd, WanVideoWrapper, WanAnimatePlus) vendor their own diffsynth and it can win the import race. This node evicts the cached copy and reimports from site-packages; if it still can't, the README's isolated subprocess mode is the nuclear option.

Honest verdict: this node is the toll booth on a research-grade pipeline. You want a 24 GB-class card to be comfortable, and the install is the hardest part of the whole pack. But the safe-install tooling is more careful than most of what's out there - clearly someone got burned by an environment cascade once and decided nobody else should have to.