ComfyUI Node

Load WiLoR

The hand specialist — but it drags MANO in with it

By ameliacode·Created 5 months ago·Updated 2 months ago· 5
Load WiLoR
    • model
    model_sourcelocal
    model_path/tmp/ComfyUI/models/wilor
    hf_token
    deviceauto

    Load WiLoR is the loader for the dedicated hand path in this pack, and it's the answer to the question you'll inevitably ask after running Body: NLF for the first time: why are the hands flat? Because NLF's body estimate doesn't include them, and WiLoR exists precisely to fix that. WiLoR (Potamias et al., 2024) is a state-of-the-art in-the-wild hand reconstructor - it detects both hands in the image and recovers their pose, which this pack converts into SMPL-X hand parameters. Wire the resulting smplx_hands into a body estimator and your flat hands become posed ones, grafted on wrist-relative so they don't melt into the body.

    Unlike the body-estimator loaders, Load WiLoR does not need an smplx_model input. It's a hand-only pipeline: hands out, no body geometry required. That makes it a bit lighter to set up, but it also means you can't use it standalone - the output only becomes useful when you feed it into Body: NLF or Full Body: Multi-HMR2.

    What you set

    • model_source - local (default) or huggingface.
    • model_path - ComfyUI/models/wilor/, holding two files: wilor_final.ckpt (the hand network) and detector.pt (the hand detector). Both come from the WiLoR HuggingFace space.
    • hf_token - gated/private repos only.
    • device - auto / cuda / cpu.

    Output is a model connection (type WILOR_MODEL) feeding Hand: WiLoR.

    The extra file nobody expects

    Here's the trap that makes this loader more annoying than its siblings: WiLoR is MANO-based, and MANO (the hand model from MPI) is registration-walled, just like SMPL-X. You need MANO_LEFT.pkl and MANO_RIGHT.pkl placed in <node>/vendor/WiLoR/mano_data/. Registration is at mano.is.tue.mpg.de. The WiLoR weight itself is CC-BY-NC-ND - and the ND part means no derivatives, so treat the whole hand path as strictly research/experimental, not as the base of a product.

    Installing

    Pack install as usual - ComfyUI Manager, search SMPLx Estimator, install, restart - or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/ameliacode/ComfyUI-SMPLx-Estimator.git
    cd ComfyUI-SMPLx-Estimator
    pip install -r requirements.txt
    python install.py
    

    install.py clones the WiLoR source into vendor/WiLoR (override with WILOR_DIR), and the requirements add the WiLoR-specific stack: ultralytics, yacs, timm, pytorch-lightning. That's the price of the detector, and it's the heaviest side-dependency set in the pack.

    Gotchas

    • The MANO files are the whole game. The most common failure is a hand estimator that loads fine but dies the moment you run it, because vendor/WiLoR/mano_data/ is empty. Check that before anything else.
    • Detection threshold lives on the estimator. conf (default 0.3) is a Hand: WiLoR parameter, not a loader one. If a hand isn't being picked up, tune there.
    • You can't get face from this. WiLoR is hands only; for facial expression you'll still want the SMIRK path.

    The setup friction is the highest in the pack (two weight files plus the MANO registration), but if you care about hands - and anyone doing full-body estimation eventually does - it's worth it. Load it once, wire it in, and your NLF results stop looking like mannequins.

    CategorySMPLx Estimator/loaders

    Inputs (4)

    NameTypeDefaultDescription
    model_sourceCOMBOlocal2 options: local, huggingface
    model_pathSTRING/tmp/ComfyUI/models/wilorlocal: folder holding the weight file(s). huggingface: leave model_path; set hf_token for gated/private repos.
    hf_tokenSTRINGHuggingFace access token.
    deviceCOMBOauto3 options: auto, cuda, cpu

    Outputs (1)

    NameTypeDescription
    modelWILOR_MODEL