👁️ Neural Pupil Tracker

Of the 14 nodes in the kanibus/kanibus pack, this is the crown jewel - and the honest way to say it is that it's the one node that does what its marketing claims. The rest of the pack is a mix of real utilities and placeholder stubs, but NeuralPupilTracker is the real deal: it runs MediaPipe's face mesh with iris refinement on every frame and extracts actual pupil positions, gaze vectors, blinks and masks you can feed downstream. If you came here from Google because "eye tracking ControlNet for WAN" sounded cool, this node is why the pack exists.

How it works

Under the hood it's MediaPipe FaceMesh with refine_landmarks=True, which gives you the 468-point face mesh plus the 4 iris landmarks per eye (indices 468–475). From there it does the actual signal processing:

• Pupil position is the centroid of each iris, then smoothed through a 6-DOF Kalman filter (KalmanFilter(dim_x=6, dim_z=2) in the source) - that's what turns jittery raw detections into usable tracking.
• Blink detection via Eye Aspect Ratio (EAR): the vertical-to-horizontal ratio of each eye, compared against blink_threshold (default 0.25).
• Saccade detection flags fast eye jumps against saccade_threshold (default 300°/s).
• 3D gaze computes a gaze vector per eye from the iris-to-eye-corner geometry, and if both eyes are tracked, a convergence point where the two gaze rays meet.
• Pupil dilation is estimated from iris landmark spread - a weak proxy, but it's there.

It also emits per-eye masks (convex regions around the eye landmark groups) as left_eye_mask and right_eye_mask. That's the output most people actually want for video-edit work: clean, per-eye binary masks at native frame resolution.

The inputs you'll set

• sensitivity (0.1–3.0, default 1) - scales thresholds up or down; raise it for low-light or blurry footage.
• smoothing (0–1, default 0.7) - how much the Kalman filter damps movement. High = steadier but laggier.
• blink_threshold - lower it if blinks aren't being caught, raise it if the eyes flicker closed in the output.
• saccade_threshold - unless you're doing gaze-behavior analysis, leave it.

previous_result takes the previous frame's EYE_TRACKING_RESULT for temporal continuity (pass it if you're looping frames yourself), and enable_3d_gaze / enable_saccade_detection / enable_pupil_dilation just toggle those features.

Outputs: tracking_result (the full EYE_TRACKING_RESULT object - pupil coords, gaze vectors, blink state, confidence), annotated_image (overlay), gaze_visualization, and the two eye masks. The masks and annotated image are standard MASK/IMAGE types, so they wire into any normal ComfyUI masking or inpainting workflow - you don't need the rest of the Kanibus pack to use them.

Installing

cd ComfyUI/custom_nodes
git clone https://github.com/kanibus/kanibus
cd kanibus    # lowercase - the README's "cd Kanibus" fails on case-sensitive filesystems
pip install -r requirements.txt   # or requirements_minimal.txt if that clashes
python install.py

Restart ComfyUI; the node lives under Kanibus/Tracking. MediaPipe downloads its face-mesh model automatically on first run (~6MB), so no separate model download needed. The README's insistence on ~5.6GB of ControlNet models doesn't apply to this node - it never loads one.

Gotchas

• The node is stateful. Kalman filters and frame counters live on the node instance, so results depend on execution order and don't reset cleanly between separate jobs. Rerun a single-frame workflow and you may see filtering carry over.
• One face max. max_num_faces=1, so it tracks a single subject.
• No face = safe defaults. It returns centered pupils, blink=open, and zero-confidence masks rather than erroring - a blank frame won't crash your workflow, it'll just silently be garbage.
• BGR/RGB conversion. Input is converted via cv2.COLOR_BGR2RGB; if your source is already RGB and something upstream is feeding it oddly, you can get subtly wrong iris geometry. Feed it normal ComfyUI IMAGE tensors and it's fine.

Real talk: if all you need is blink-aware eye masks or a gaze point to drive an interactive workflow, this node genuinely works and runs fast. It's worth installing the pack for this one node alone - just ignore the marketing around it.