Wan Gaze — ETH-XGaze Post-Processor (V2)

WanGazeETHXGazeV2 is a post-processor with one job: replace the gaze vectors in your pose_data's iris data with predictions from the ETH-XGaze ResNet-50 (ECCV 2020), a model that claims ~2.5° mean angular error in the wild. The default gaze engines in this pack are good (L2CS-Net at ~10.4°, or the MediaPipe blendshape path); this node is the upgrade path if you want the accuracy and are willing to fetch a checkpoint and clone a repo to get it.

This is the most "lab equipment" node in the pack, and it behaves like it: it's only registered if the ETH-XGaze prerequisites exist, and it silently falls back to the previous engine's output when they're missing. The tooltip says "on any missing prerequisite the engine silently keeps the previous engine's output" - which is a fancy way of saying you should read the requirements twice before wiring it in.

Setup - this one actually needs work

The two-step setup is the main cost:

1. Clone the ETH-XGaze repo so face_model.txt and model.py are available:

   git clone https://github.com/xucong-zhang/ETH-XGaze third_party/ETH-XGaze

2. Download the checkpoint epoch_24_ckpt.pth.tar into ComfyUI/models/ethxgaze/. It's the standard ETH-XGaze pretrained weights file.

The node's checkpoint dropdown auto-discovers from models/ethxgaze/ and third_party/ETH-XGaze/ckpt/; there's a checkpoint_path_override if you want to point at an absolute path instead. If the dropdown shows "<none - drop epoch_24_ckpt.pth.tar in models/ethxgaze/>", that's the checklist item you haven't done.

How it works

For each frame it takes the face crop (the node needs images - the same RGB stack the pose_data was computed from), runs a pose-normalized 224×224 face warp, and feeds the result through the ETH-XGaze gaze_network in batches of batch_size (default 8). The pose normalization is the whole trick: it removes head roll and fixes the camera distance, so the regressor sees a front-facing face and the error stays low on tilted heads.

Inputs: pose_data, images, and the checkpoint dropdown. The useful ones beyond that are blend (0–1, default 1) - at 0 you keep the original gaze, at 1 it's full ETH-XGaze, in between you can ease the new model in and compare - and device (auto / cuda / cpu). Outputs: the same pose_data with replaced gaze vectors, plus an info string.

The honest take

Is it worth it? Depends what you're driving. For a talking-head Wan Animate scene where the subject mostly looks at camera, the default gaze engine is fine and this node is friction you don't need. Where ETH-XGaze earns its keep is off-axis, tilted-head footage - the pose-normalized warp is meaningfully better there, and gaze accuracy feeds directly into how convincing the character's eyes look in the render, because Wan Animate's face encoder reads crop pixels and gaze direction is part of that signal. The silent-fallback behavior is the real trap: if your checkpoint path is wrong, the node doesn't fail loudly - it just keeps whatever gaze was already there, and you'll never know why your upgrade did nothing.

Install of the pack itself is the shared path (Manager search "WanAnimatePreprocessV2", or clone + pip install -r requirements.txt); this node additionally needs the two manual steps above plus the optional ETH-XGaze instructions in requirements.txt.