ComfyUI Node
Pose and Face Detection (V2)
Run YOLO person detection + ViTPose 2D keypoints + (optional) MediaPipe FaceMesh on a video tensor. Produces the full pose/face/iris bundle required by Wan 2.2 Animate Character Replacement workflows. Wan-Animate fidelity notes (spec 2.5-2.7, workflow-level — not something this node can enforce for you): 2.5 Wan-Animate splices long generations in ~78-frame segments with 1-5 frame temporal handoffs (WanVideoAnimateEmbeds.frame_window_size, default 77); a brief microexpression (often only 2-4 frames) landing exactly on a segment boundary risks being smoothed by the discard/resume splice. If a specific expression must survive, check where it falls relative to your frame_window_size and shift the cut (or duration) if needed. 2.6 Feed this node NATIVE framerate footage. Don't downsample fps upstream (e.g. a LoadVideo 'force_rate' below the source fps) — the face branch's causal 1D-conv temporal downsampling further compresses an already-fps-reduced brief expression, and a 2-4 frame microexpression can be lost entirely before it ever reaches this node. 2.7 If your workflow globally autocasts to fp8/fp16, the face encoder's small-magnitude motion-basis deltas are more vulnerable to quantization noise than the body/pose branch. As of this writing Kijai's ComfyUI-WanVideoWrapper has no per-module precision override for the Wan-Animate face branch specifically — if you need this, load the full checkpoint in fp16/fp32 rather than an fp8 quant.
Pose and Face Detection (V2)
- model
- images
- bbox_override
- hires_images
- retarget_image
- pose_data
- face_images
- key_frame_body_points
- bboxes
- face_bboxes
- iris_data
- debug_image
- right_pupil_xy
- left_pupil_xy
- lip_openness_ratio
- restore_info
- face_cfg_scale
- face_images_512
- expression_coeffs_json
◄width832►
◄height480►
◄detection_threshold0.05►
◄pose_threshold0.30►
◄use_clahetrue►
◄clahe_clip_limit2.0►
◄clahe_grid_size8►
◄detect_gamma1.00►
◄detect_white_balancefalse►
◄detect_denoise0.00►
◄detect_sharpen0.00►
◄detect_saturation1.00►
◄use_blur_for_posefalse►
◄blur_radius5►
◄blur_sigma2.0►
◄use_face_smoothingtrue►
◄face_smoothing_strength0.60►
◄use_constant_face_boxtrue►
◄face_crop_scale1.30►
◄face_box_size_px0►
◄use_iris_smoothingtrue►
◄iris_smoothing_strength0.40►
◄iris_smoothing_methodone_euro►
◄iris_one_euro_min_cutoff1.00►
◄iris_one_euro_beta0.05►
◄gaze_lock_eyestrue►
◄gaze_lock_strength0.70►
◄use_mediapipe_facetrue►
◄use_blendshape_gazetrue►
◄gaze_one_euro_min_cutoff1.70►
◄gaze_one_euro_beta0.30►
◄gaze_max_yaw_deg30►
◄gaze_max_pitch_deg25►
◄crop_modedefault►
◄frame0_cx-1►
◄frame0_cy-1►
◄frame0_size0►
◄keyframes_json[]►
◄smoothing_methodone_euro►
◄crop_one_euro_min_cutoff1.00►
◄crop_one_euro_beta0.05►
◄crop_gaussian_window7►
◄crop_safety_margin1.12►
◄crop_size_one_euro_beta0.20►
◄crop_containment_checktrue►
◄crop_containment_tolerance4►
◄auto_smoothing_methodlegacy_ema►
◄preserve_face_aspectfalse►
◄force_eyes_open0.00►
◄eye_open_modeblinks_only►
◄eye_open_blink_ear0.18►
◄eye_align_modedefault►
◄eye_y_fraction0.30►
◄face_cfg_scale1.0►
◄gaze_enginel2cs_gaze360►
◄gaze_kalman_meas_std_deg3.0►
◄gaze_kalman_process_std0.80►
◄gaze_fps30►
◄gaze_calibration_frame-1►
◄apply_gaze_to_face_imageoff►
◄au_amplify1.00►
◄au_amplify_neutral_frame0►
◄export_expression_coeffsfalse►
◄landmark_overrides_json{}►
◄face_resize_filtermitchell►
◄face_srnone►
◄face_sr_model►
◄face_sr_stabilise0.50►
◄use_fluxfalse►
◄flux_kontext_path►
CategoryWanAnimatePreprocess_V2
Inputs (75)
| Name | Type | Default | Description |
|---|---|---|---|
| model | POSEMODEL | From ONNX Detection Model Loader (V2). | |
| images | IMAGE | Video frames as an IMAGE batch (B,H,W,C float [0,1]). | |
| width | INT | 83264–2048 | Target canvas width (px) used for retarget math. Match your Wan 2.2 latent size. |
| height | INT | 48064–2048 | Target canvas height (px). Match your Wan 2.2 latent size. |
| detection_threshold | FLOAT | 0.050–1 | YOLO confidence threshold. Lower = more permissive person detection. |
| pose_threshold | FLOAT | 0.300–1 | Per-keypoint score threshold. Below this a keypoint is treated as missing. |
| use_clahe | BOOLEAN | true | Apply CLAHE contrast enhancement for pose detection. |
| clahe_clip_limit | FLOAT | 2.00.5–8 | CLAHE contrast-limit. Higher = stronger local contrast, which helps a flat/hazy or backlit shot but starts amplifying grain. 2.0 is the long-standing default; try 3-4 for genuinely flat footage. Only used when use_clahe is on. |
| clahe_grid_size | INT | 81–16 | CLAHE tile grid (NxN). Smaller = more global/gentler; larger = more aggressively local, which can rescue a face lost in shadow but may introduce tile seams. Only used when use_clahe is on. |
| detect_gamma | FLOAT | 1.000.1–3 | Gamma applied to the DETECTOR's input only. >1 lifts shadows (a face crushed into darkness becomes detectable), <1 pulls down blown highlights. Applied BEFORE CLAHE so there is signal in range for CLAHE to equalise. 1.0 = off. |
| detect_white_balance | BOOLEAN | false | Grey-world white balance on the DETECTOR's input only. Equalises the per-channel means to remove a colour cast (tungsten, underwater, heavy LUT). Skin tone drifting off-neutral is a common cause of low face-detection confidence. Runs first, so CLAHE is not amplifying a cast. |
| detect_denoise | FLOAT | 0.000–1 | Edge-preserving (bilateral) denoise on the DETECTOR's input only. For grainy/high-ISO or heavily compressed footage where noise costs keypoint precision. Bilateral rather than blur so ViTPose keeps the edges it localises from. Runs before sharpen so noise is never sharpened. 0 = off. |
| detect_sharpen | FLOAT | 0.000–2 | Unsharp-mask amount on the DETECTOR's input only. Recovers landmark precision on soft/out-of-focus or upscaled footage. Runs after denoise. Overdoing it creates halos that pull landmarks toward edges — 0.3-0.6 is usually plenty. 0 = off. |
| detect_saturation | FLOAT | 1.000–2 | Chroma scale on the DETECTOR's input only. Slightly boosting saturation can separate skin from a similarly-lit background; dropping toward 0 makes detection effectively luma-only, which occasionally helps on heavily colour-graded footage. 1.0 = off. |
| use_blur_for_pose | BOOLEAN | false | Apply Gaussian blur internally for YOLO and ViTPose BEFORE detection. Bug-fix (default was True): this softens the exact edges/fine detail ViTPose needs for keypoint precision, producing a visibly blurrier preview and a less accurate skeleton for every user until they discovered and disabled it. Only enable this for genuinely noisy/grainy source footage. |
| blur_radius | INT | 51–20 | Gaussian blur kernel radius applied to the face mask edge to soften the boundary. Higher = wider feather. Kernel size = radius*2+1 px. |
| blur_sigma | FLOAT | 2.00.1–5 | Gaussian blur sigma (standard deviation) for the face mask feather. Higher sigma = softer falloff. Tune together with blur_radius. |
| use_face_smoothing | BOOLEAN | true | Smooth face bounding box center over time. |
| face_smoothing_strength | FLOAT | 0.600–1 | Higher = more smoothing |
| use_constant_face_box | BOOLEAN | true | Keep a constant pixel size face crop; position adapts. |
| face_crop_scale | FLOAT | 1.301–3 | AREA expansion of the face box, passed straight to get_face_bboxes. 1.3 is the value Wan2.2's own process_pipepline.py uses at both call sites, so 1.3 = reference-exact. LOWER (1.1-1.2) crops tighter, which puts MORE pixels on the face after the 512 resize and is the single most effective knob for micro-expression detail; too low and a head turn can clip the jaw/ear. HIGHER gives more headroom and safety at the cost of face resolution. Applies to every crop_mode. |
| face_box_size_px | INT | 00–1024 | Side of the constant-size face crop, in SOURCE pixels, for crop_mode=auto (with use_constant_face_box) and jitterless. 0 = AUTO (recommended): the side is derived from the median DETECTED face box across the clip, then held constant for every frame. You get the reference pipeline's face-tight framing - the face FILLS the tile - while the size still never breathes, which is the whole point of these modes. A fixed value is an ABSOLUTE pixel size and is almost always wrong unless you know your footage: the old 512 default clamps to min(width,height), so on an 832x480 clip it cut a 480px window around a ~125px face. The face then filled about a quarter of the tile and the rest was background, throwing away roughly 3/4 of the resolution a micro-expression needs and making the subject look off to one side. Set a fixed value only to lock a specific framing across separate renders. |
| use_iris_smoothing | BOOLEAN | true | Temporally smooth iris pixel positions across frames. Reduces per-frame jitter that Wan 2.2 Animate's face encoder picks up and reproduces as wobbly gaze. |
| iris_smoothing_strength | FLOAT | 0.400–1 | EMA mix weight when iris_smoothing_method='ema'. Higher = more smoothing, more lag. Ignored for one_euro / none. |
| iris_smoothing_method | COMBO | one_euro | Iris pixel-position smoother. one_euro = adaptive low-pass (Casiez 2012, recommended). ema = legacy first-order; tweak via iris_smoothing_strength. none = raw per-frame positions. |
| iris_one_euro_min_cutoff | FLOAT | 1.000.05–10 | One-euro min cutoff (Hz) for iris pixel coords. Lower = stronger jitter rejection on near-static eyes (small saccades preserved). |
| iris_one_euro_beta | FLOAT | 0.050–5 | One-euro speed coefficient for iris pixel coords. Higher = filter relaxes faster on quick eye movements; lower = stronger steady-state smoothing. |
| gaze_lock_eyes | BOOLEAN | true | Couple left & right eye gaze so they always look in the SAME direction. Both eyes' yaw/pitch are blended toward their per-frame average. Single most effective fix for the 'eyes pointing different directions' artefact in Wan 2.2 Animate output. |
| gaze_lock_strength | FLOAT | 0.700–1 | How strongly to pull each eye toward the shared average. 0 = independent (legacy). 1 = perfectly conjugate (both eyes always parallel). 0.7 keeps a touch of natural convergence/divergence. |
| use_mediapipe_face | BOOLEAN | true | Use MediaPipe FaceMesh (478 pts incl. iris/lips) to override face landmarks. Falls back to ViTPose pupil voting if MediaPipe is missing or fails on a frame. |
| use_blendshape_gaze | BOOLEAN | true | Use MediaPipe FaceLandmarker (Tasks API) blend shapes for production-grade per-eye yaw/pitch in radians. Head-pose-corrected by training. Auto-downloads face_landmarker.task (~3MB) on first run. Falls back to legacy 2D iris-offset gaze if disabled or unavailable. |
| gaze_one_euro_min_cutoff | FLOAT | 1.700.05–10 | One-euro filter base cutoff frequency (Hz). Lower = more aggressive jitter rejection at the cost of slight lag. 1.7 is a good default for 24-30 fps gaze. |
| gaze_one_euro_beta | FLOAT | 0.300–5 | One-euro filter speed coefficient. Higher = filter relaxes faster on quick saccades, preserving responsiveness; lower = stronger smoothing during fast moves. |
| gaze_max_yaw_deg | FLOAT | 305–60 | Saturation yaw angle in degrees that corresponds to blend shape value 1.0. 30° covers the comfortable physiological range; raise for more dramatic eye motion. |
| gaze_max_pitch_deg | FLOAT | 255–60 | Saturation pitch angle in degrees that corresponds to blend shape value 1.0. 25° covers the comfortable physiological range. |
| crop_mode | COMBO | default | How the face crop box is built. Four modes, no more - the ones that measured well and the ones you asked to keep. default = the REFERENCE behaviour, byte for byte with Wan2.2's own process_pipepline.py: per-frame face-tight box, no smoothing of anything. Measured 1.09px of face wander inside the 512 tile and 100% face-fill. This is the safe choice and the one to A/B against. expression_lock = the reference box with the centre taken RAW per frame and only the box SIZE stabilised, so the tile stops breathing. Measured 1.18px wander, 100% fill - statistically the same as default, with a steadier tile size. jitterless = locked constant-size crop, Mocha-style planar hold. Steadiest tile SIZE of all, at the cost of face-fill when the subject moves toward or away from camera. auto = legacy motion-adaptive smoothing with an optional constant-size box. The most forgiving on very jittery handheld, the least faithful on subtle expression. RETIRED: central_face and reference_smooth. Both still RUN if a saved workflow selects them, but they are no longer offered - central_face crops eyebrow-to-mouth only, which starves an already small face, and reference_smooth filters the crop CENTRE, which lets the face drift inside the tile (measured 26-61px on a pan) and spends Wan-Animate's 20-number face budget on rigid motion instead of expression. NOTE: the mode matters far less than the face RESOLUTION. If your face box is under ~160px the tile is mostly invented pixels and no mode fixes that - wire a full-res plate to hires_images. |
| frame0_cx | INT | -1-1–8192 | Frame 0 anchor center X in pixels. -1 = use detected face center on frame 0. Used only when crop_mode=jitterless. |
| frame0_cy | INT | -1-1–8192 | Frame 0 anchor center Y in pixels. -1 = use detected face center on frame 0. |
| frame0_size | INT | 00–4096 | Locked square crop size in pixels (used for the entire clip). 0 = fall back to face_box_size_px. |
| keyframes_json | STRING | [] | JSON list of per-frame overrides: [{"frame":N, "cx":X, "cy":Y, "size":S?}, ...]. Frames between key-frames are linearly interpolated. size is optional; if omitted the locked size is kept. |
| smoothing_method | COMBO | one_euro | Center-trajectory filter. one_euro = jitterless adaptive low-pass (recommended). ema = legacy motion-adaptive EMA. gaussian = fixed-window 1D blur. none = raw. |
| crop_one_euro_min_cutoff | FLOAT | 1.000.05–10 | One-euro min cutoff (Hz) for crop center. Lower = stronger jitter rejection. |
| crop_one_euro_beta | FLOAT | 0.050–5 | One-euro speed coefficient for crop center. Higher = filter relaxes faster on quick motion. |
| crop_gaussian_window | INT | 73–51 | Window size (odd) for the Gaussian temporal blur of the crop center. |
| crop_safety_margin | FLOAT | 1.121–2 | Inflate the crop by this factor before smoothing so filter lag, yaw-foreshortened detections and expression-driven bbox growth cannot clip the face. 1.0 = no margin (old behaviour). Applies to both 'auto' and 'jitterless'. If crop_containment_check reports corrections on more than a handful of frames, raise this toward 1.15-1.20 rather than fighting it downstream. |
| crop_size_one_euro_beta | FLOAT | 0.200–2 | One-euro beta for the crop SIZE trajectory, separate from crop_one_euro_beta (which is the CENTER's). Position wants heavy damping to kill detector jitter; scale wants to follow real zoom/approach or the crop under-sizes mid-move. Only used when the size is allowed to vary (crop_mode='auto', or jitterless with explicit key-frame sizes) — a locked jitterless size ignores it by definition. |
| crop_containment_check | BOOLEAN | true | HARD per-frame guarantee that the actual detected face bbox ends up inside the final crop. After smoothing, any frame whose face escapes the crop is corrected. In 'jitterless' the correction SHIFTS the crop (the exact-size lock is preserved); growing would silently break the lock, so a face genuinely larger than the locked size is reported in the log instead — that means face_box_size_px / crop_safety_margin is too small for the shot. In 'auto' the crop may grow. Correction counts are logged. |
| crop_containment_tolerance | INT | 40–128 | Extra pixels of slack required around the detected face bbox when crop_containment_check tests containment. |
| auto_smoothing_method | COMBO | legacy_ema | Which filter crop_mode='auto' uses. 'legacy_ema' keeps auto's original bespoke EMA byte-for-byte (the default, so existing workflows are untouched); the others route auto through the same shared filters jitterless uses, honouring crop_one_euro_* / crop_gaussian_window. Ignored unless crop_mode='auto'. |
| preserve_face_aspect | BOOLEAN | false | OFF (default) = square crop, which is what Kijai's WanVideoWrapper needs: WanVideoAnimateEmbeds re-resizes anything that is not already 512x512 with common_upscale(..., 'center'), and that CENTER-CROPS to square first, so a non-square tile would have its top and bottom cut off. This node always emits 512x512 so the wrapper passes it straight through. ON = build the crop at the per-frame face-box aspect before the 512 resize. EXPERIMENTAL and NOT what the reference does: Wan2.2's process_pipepline.py simply does get_face_bboxes -> frames[y1:y2, x1:x2] -> cv2.resize(512,512) per frame, with no aspect tracking, no locked size and no smoothing. For the reference behaviour exactly, use crop_mode='default'. |
| force_eyes_open | FLOAT | 0.000–1 | Force closed/blinking eyes open. 0 = off (default). 1 = fully open to a natural EAR of ~0.30; intermediate values blend. This REACHES THE MODEL because it pairs with DrawViTPoseV2's apply_pose_edits_to_face warp: Wan-Animate's face conditioning is 100%% pixel-driven (landmarks only place the crop, the LIA motion encoder reads raw crop pixels), so this node writes opened-eye LANDMARKS and DrawViTPoseV2 warps the actual crop PIXELS to match — using each frame's own crop as the source, so identity, head pose, mouth and lighting are preserved and only the eye aperture changes. Wire face_images/face_images_512 -> DrawViTPoseV2.face_images and leave apply_pose_edits_to_face='warp' (the default) or this does nothing visible. Only ever opens, never closes; eye corners stay fixed so the warp stays local. |
| eye_open_mode | COMBO | blinks_only | Which frames force_eyes_open targets. 'blinks_only' = only frames whose measured Eye-Aspect-Ratio falls below eye_open_blink_ear (keeps natural performance, removes blinks). 'all_frames' = whole-shot override, for when the subject squints throughout. |
| eye_open_blink_ear | FLOAT | 0.180.01–0.4 | Eye-Aspect-Ratio below which a frame counts as a blink for eye_open_mode='blinks_only'. A natural open eye is ~0.28-0.35, a full blink ~0.05-0.15. Raise toward 0.22 to also catch heavy-lidded frames. |
| eye_align_mode | COMBO | default | Wan-Animate paper recommendation #1: 'eye_upper_third' vertically shifts the face crop so eyes land at the upper third of the 512x512 face encoder input. The encoder reads holistic face appearance, so consistent eye placement directly improves gaze fidelity. 'default' keeps legacy bbox center. |
| eye_y_fraction | FLOAT | 0.300.1–0.6 | Target eye row as a fraction of crop height (0.30 = upper third). Only used when eye_align_mode = 'eye_upper_third'. |
| face_cfg_scale | FLOAT | 1.01–10 | Wan-Animate paper Sec. 4.3 names CFG on the face-conditioning branch as one lever for finer expression control, BUT Kijai's ComfyUI-WanVideoWrapper has no separate face-CFG input to wire this into — wiring it nowhere is a dead passthrough. The wrapper instead exposes a STRONGER, more direct lever for exactly this purpose (spec 2.2: 'a raw face-adapter block-scale... changes contribution before guidance math rather than after'): WanVideoAnimateEmbeds.face_strength (default 1.0, try 1.5-2.5 for stronger expression adherence). Use that widget on your WanVideoAnimateEmbeds node instead. This FLOAT output is kept for any sampler that DOES expose a genuine face-CFG input and for forward-compat; 1.0 = no-op. |
| gaze_engine | COMBO | l2cs_gaze360 | Per-eye gaze yaw/pitch engine. DEFAULT is now l2cs_gaze360 (GPU/CUDA, auto-downloads ~100MB once) so gaze runs on the GPU; blendshape_* are the CPU-only fallbacks. * iris_geometric (NEW, deterministic): MEASURES the MediaPipe iris centre inside the eye aperture (corner-to-corner, lid-to-lid) instead of estimating gaze with a NN — no per-person appearance bias, per-eye output, blink-gated, composed with the solvePnP head pose + Kalman like blendshape_head_corrected. Best fidelity for animation retargeting (the character's eyeballs copy the performer's iris positions). Pure CPU math, no downloads. * blendshape_head_corrected (DEFAULT, recommended): MediaPipe ARKit blend shapes + solvePnP head pose + Kalman temporal smoother. Eye-in-head rotation is composed with the head rotation so the rendered arrow tracks rotated heads. Pure numpy + cv2, no downloads. * blendshape_only: legacy May-2026 shipped behavior; eye-in-head only, no head composition. * l2cs_gaze360: L2CS-Net (MIT) ResNet50 trained on Gaze360. ~10.4° MAE but robust to extreme poses (recommended for Wan-Animate character scenes). One-time ~100MB weight download to ComfyUI/models/gaze/. * l2cs_mpiigaze: L2CS-Net MPIIGaze variant. ~3.9° MAE but calibrated only for near-portrait subjects. * pose_normalized_resnet50: Highest-accuracy path. Pipeline = solvePnP head pose -> analytical pose-normalized 224x224 face warp (head roll removed, camera distance fixed at 600 mm) -> ResNet50+Linear(2048,2) gaze regressor -> de-rotate output back to camera frame. Major accuracy gain on tilted / off-axis heads. The normalization warp is a clean-room implementation of the 2018 ETRA paper's published equations and ships with this pack (Apache-2.0). The ResNet50 checkpoint is NOT bundled — place a community-released gaze-trained ResNet50 weight file at <ComfyUI>/models/gaze/pose_normalized_resnet50.pth.tar to enable this engine. Note: those community checkpoints are typically released under CC BY-NC-SA 4.0 (non-commercial); you are responsible for confirming the licence of any weights you install matches your use case. If the file is missing the node automatically falls back to l2cs_gaze360. * ethxgaze: ETH-XGaze ResNet-50 (ECCV 2020, ~2.5° in-the-wild MAE). Post-processes iris_data using pose-normalised 224x224 face crops + the official gaze_network. Requires (a) the third_party/ETH-XGaze/ repo cloned for face_model.txt + model.py and (b) checkpoint `epoch_24_ckpt.pth.tar` placed in `ComfyUI/models/ethxgaze/`. On any missing prerequisite the engine silently keeps the previous engine's output. |
| gaze_kalman_meas_std_deg | FLOAT | 3.00.1–20 | Kalman measurement noise (degrees). Higher = trust the model less and lean on the velocity model more — smoother. Used by blendshape_head_corrected and l2cs_* engines. |
| gaze_kalman_process_std | FLOAT | 0.800.05–5 | Kalman process noise (rad/s). Roughly the expected saccade velocity scale. Higher = filter reacts faster to genuine motion but jitters more. |
| gaze_fps | FLOAT | 301–240 | Video fps used by the Kalman dt. Set to match your source clip; affects velocity coupling, not absolute angles. |
| gaze_calibration_frame | INT | -1-1–999999 | W7-G2 per-shot gaze calibration (iris_geometric engine only). Set this to a frame index where the subject looks STRAIGHT AT THE CAMERA; the measured eye-in-head angles on that frame become the zero reference for the whole shot, removing per-person eye-shape bias (the last few degrees of error no model can fix). -1 = off. |
| apply_gaze_to_face_image | COMBO | off | Move real iris pixels in the face crop to match the computed gaze. DEFAULT IS OFF, deliberately. This is a Delaunay warp of the actual OUTPUT pixels and it is only as good as the gaze estimate driving it. If the estimate is wrong it DAMAGES face_images rather than helping. Turn it on only after checking the gaze arrows are correct on your footage AND you actually want to override the performer's own eyes. Note gaze reaches Wan-Animate ONLY through these pixels - the pose conditioning image is a body skeleton with five coarse head dots and no iris. With this off, the gaze in face_images is whatever the camera saw, which is normally what you want. |
| au_amplify | FLOAT | 1.001–1.5 | Wan-Animate spec 2.3: the face encoder compresses to a small fixed-capacity motion-basis vector, so a genuinely subtle real microexpression can sit near the compression noise floor. This pushes each frame's detected face landmarks a bit FURTHER along the direction they already moved from the neutral reference frame (au_amplify_neutral_frame) — amplifying REAL, DETECTED motion so more of it survives compression; it never synthesizes anything that wasn't already measured. 1.0 = off (default). 1.15-1.3 is the range the paper's own architecture analysis suggests; values are capped at 1.5 since the correction is only a 2D (eye-line roll+scale) head-pose approximation, not a full 3D one — the discrepancy grows with head yaw/pitch, so keep this modest for non-frontal shots. Delivered via the same Delaunay real-pixel warp as 'warp' gaze mode; gated by the same blur/quality check; on any per-frame failure that frame is left unamplified. |
| au_amplify_neutral_frame | INT | 00–999999 | Frame index to use as the NEUTRAL reference for au_amplify — pick a frame where the subject's expression is relaxed/neutral (Wan-Animate spec 2.4: an already-tense or asymmetric reference eats into the same motion-basis budget the target microexpression needs). Ignored when au_amplify=1.0. |
| export_expression_coeffs | BOOLEAN | false | Wan-Animate spec 3.1 (closed-loop critic, foundation): export the 'expression_coeffs_json' output — per-frame ARKit-52 blendshapes measured from this run's iris_data. Off by default (no extra cost when unused). Run this node once on the source driving video and once on the Wan-Animate generated output, then wire the source run's expression_coeffs_json into DrawViTPoseV2.reference_expression_coeffs_json for a per-AU fidelity report. |
| bbox_overrideopt | BBOX | Optional external BBOX for the frame-0 anchor. Highest priority; overrides frame0_cx/cy/size widgets. | |
| landmark_overrides_jsonopt | STRING | {} | Manual body-keypoint corrections from the Pose editor. Shape: {"<frame>": {"<jointIdx>": [x_px, y_px], ...}, ...} in SOURCE pixels. Written by the viewer's Edit mode — drag a joint to fix a mis-detection and the correction flows through retargeting into pose_data AND the rendered pose images (not just the preview). Leave as {} for pure detection. |
| face_resize_filteropt | COMBO | mitchell | Resampling filter for the face crop -> 512 resize. Nuke's filter set, same names, so a compositor can reason about it with the vocabulary they already use. MEASURED on the real 46px-face case, resizing 46 -> 512 -> 64 and comparing against 46 -> 64 direct (lower error = less information mangled, overshoot = ringing): cubic err 0.00297 overshoot 0.0000 mitchell err 0.00311 overshoot 0.0000 <- DEFAULT keys err 0.00329 overshoot 0.0000 lanczos4 err 0.00715 overshoot 0.0001 rifman err 0.01007 overshoot 0.0312 sinc4 err 0.02274 overshoot 0.0804 The previous hardcoded cv2 LANCZOS4 measured 0.00721 - about 2.4x worse than mitchell. WHY: a filter rings when its kernel goes NEGATIVE, overshooting at hard edges. That overshoot is invented structure the 20-number face encoder cannot tell from real texture, and it is why a crop taken to 512 and sampled back down does not match the original sampled down directly - the ringing does not cancel. cubic / parzen / notch / impulse cannot ring (no negative lobes). mitchell has small negative lobes and measured zero overshoot here - Mitchell and Netravali's own paper concludes B=C=1/3 is the best blur-versus-ringing compromise, which is why it is the default. keys / simon / rifman / lanczos / sinc are progressively sharper and ring progressively harder. DOWNSIZING: prefer mitchell, or parzen if the plate is noisy - ringing survives into the latent as hard fringes, and softness does not. |
| face_sropt | COMBO | none | Super-resolve the face crop BEFORE it is resized to the 512x512 the encoder needs. Why it exists: the tile is always 512, but the region it is cut from is whatever the plate gives. A 46px face box means an ELEVEN-times upscale, so almost everything the model reads is invented by a resize filter. HeadsUp! (arXiv:2510.09924) discards training faces under 64px interocular (~160px face box) as too small to learn from - the node logs where your shot sits. none = plain resize (previous behaviour). lanczos = deterministic baseline. Invents nothing, so if a real SR model does not beat this on your footage, that model is only adding hallucination and cost. comfy_upscale = any ESRGAN-family model in ComfyUI/models/upscale_models, named in face_sr_model. SR runs on the NATIVE crop, before the 512 resize. Order matters: Lanczos is a windowed sinc with negative lobes, so it rings on hard edges; running SR after that would just sharpen the ringing. It is also why a crop taken to 512 and sampled back down does not match the original sampled down directly - the ringing does not cancel. PREFER hires_images if you have a full-res plate. Real pixels always beat invented ones. |
| face_sr_modelopt | STRING | Filename of the upscale model in ComfyUI/models/upscale_models, used only when face_sr=comfy_upscale. Errors naming what is installed if it cannot be found. | |
| face_sr_stabiliseopt | FLOAT | 0.500–1 | How hard to stabilise SR detail over time. Per-frame SR hallucinates independently, so texture BOILS - and boiling is exactly the high-frequency per-frame noise Wan-Animate's 20-number motion code cannot tell from real motion. Hallo2 (arXiv:2410.07718) finds SR only helps expression fidelity when paired with temporal alignment. Only the DETAIL layer is filtered; the base is left alone so real motion is never smeared. Measured on a moving subject: 0.5 -> flicker -37% with 101% of real motion preserved; 0.7 -> -49% / 87%; 1.0 -> -58% / 75%. 0.5 is the last value that costs nothing in motion. Raise it only if a shot still boils. |
| hires_imagesopt | IMAGE | OPTIONAL, and the single biggest quality lever in this node. Wire your FULL-RESOLUTION plate here (the EXR/source sequence). Detection still runs at the working resolution because that is what ViTPose wants and it is fast, but the face TILE is cut from this hi-res source instead. Why it matters more than any crop_mode: on an 832x480 plate a face box measures about 46px, and that tile is upscaled ELEVEN times to reach the 512x512 the encoder needs - 99.2% of what the model reads is invented, and an eyeball at that scale is roughly 4 pixels across. There is no iris in 4 pixels. Every crop_mode was cropping the same starved image, which is why none of them fixed eye direction or micro-expression detail. Same framing, real pixels: 46px becomes 106px from a 1080p plate, 212px from 4K. Ignored (with a warning) if it is not larger than the working plate. | |
| retarget_imageopt | IMAGE | Optional reference image of the TARGET character. When connected, the detected driver pose is RETARGETED onto this reference's body proportions and position (the same retarget V1 had): the reference's pose is detected, then get_retarget_pose maps the driver's motion onto it. Leave unconnected for straight detection (no retarget). | |
| use_fluxopt | BOOLEAN | false | Enhanced retargeting via FLUX.1-Kontext-dev (Wan 2.2 Animate's third retarget mode). When ON with retarget_image connected, FLUX normalizes the reference AND the first template frame to a standard front-facing pose BEFORE retargeting, so retargeting starts from a neutral instead of carrying a 3/4-profile or head-tilt into the output. Recommended ONLY when the reference character is NOT front-facing; for a front-facing reference, basic retarget (use_flux off) is enough. Needs the FLUX.1-Kontext-dev model — set flux_kontext_path. The authors' caveat: FLUX.1-Kontext-dev has limited capability, consistency is not guaranteed; check the intermediate edited frames. |
| flux_kontext_pathopt | STRING | Path to the FLUX.1-Kontext-dev diffusers checkpoint FOLDER. Download from https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev. Empty = look the model up via ComfyUI's folder_paths ('flux' / 'checkpoints' / 'unet' keys, in that order). Raises a clear error naming what was searched if nothing is found — never silently disables use_flux. |
Outputs (14)
| Name | Type | Description |
|---|---|---|
| pose_data | POSEDATA | Per-frame pose+face+iris dict bundle. Feed into Draw ViT Pose (V2). |
| face_images | IMAGE | Cropped face IMAGE batch suitable for face-id encoders. |
| key_frame_body_points | STRING | Key-frame body points as JSON string (debug). |
| bboxes | BBOX | Per-frame body BBOX list. |
| face_bboxes | BBOX | Per-frame face BBOX list. |
| iris_data | STRING | Iris/gaze JSON dump (debug). |
| debug_image | IMAGE | Annotated debug IMAGE batch (skeleton overlay). |
| right_pupil_xy | STRING | Right pupil pixel xy as JSON (per frame). |
| left_pupil_xy | STRING | Left pupil pixel xy as JSON (per frame). |
| lip_openness_ratio | FLOAT | Mouth-open scalar list (0=closed, 1=wide). |
| restore_info | FACE_RESTORE_INFO | Per-frame {x1,y1,x2,y2,size,frame_shape} dict for paste-back nodes. |
| face_cfg_scale | FLOAT | CFG scale for the face conditioning input. Wire into the Wan-Animate sampler's face CFG. 1.0 = CFG off (paper default). |
| face_images_512 | IMAGE | Cropped face IMAGE batch force-resized to 512x512 (bilinear). Pre-shaped for the Wan 2.2 Animate face encoder; wire directly without an extra Resize node. |
| expression_coeffs_json | STRING | Wan-Animate spec 3.1: per-frame ARKit-52 blendshapes measured from this run's iris_data, as {fps,names,frames:[{frame,blendshapes}]}. Run this node on BOTH the source driving video and the Wan-Animate GENERATED output, then wire this output from the SOURCE run into DrawViTPoseV2.reference_expression_coeffs_json (with the GENERATED run's pose_data wired into DrawViTPoseV2.pose_data as usual) to get a per-AU fidelity report. Empty '{}' when export_expression_coeffs=False (default) or no MediaPipe blendshapes were captured this run. |