Gaze Detection Video

Same idea as Gaze Detection, applied to every frame of a video. Gaze Detection Video takes a batch of frames, finds the faces in each one, and draws the same rainbow boxes and fading gaze arrows - then hands you the whole annotated batch back. In the included example workflow it sits between VHS_LoadVideo and VHS_VideoCombine, which is the natural place for it: video in, annotated frames out, reassembled into a file. If you've ever wanted to know where the people in a clip are looking, this is the shortest path in ComfyUI.

How it works

The mechanism is deliberately simple, and that simplicity cuts both ways. It loops over the frame batch - video arrives as an IMAGE tensor where the first dimension is frame count - and runs the exact same per-image pipeline as its single-frame sibling on each frame, with a ComfyUI progress bar tracking it in the UI. Then it stacks the annotated results back into one tensor.

That means there's zero temporal logic. No tracking, no smoothing, no "the same person moved a bit between frames, keep their color." Each frame is analyzed in isolation and each face is re-detected from scratch. On a stable shot with clear faces that's fine. With motion, quick cuts, or occlusion, the boxes can flicker and the arrows can jump around - it looks like a genuinely great party trick and like the first draft of a real product, depending on your footage.

Inputs

• model - the MOONDREAM_MODEL from the pack's (Down)Load Moondream Model node. Same as the image node: you need the loader wired in, on cuda if you have the VRAM.
• video - an IMAGE tensor holding the frame batch. Note the type says IMAGE, not VIDEO - ComfyUI represents video as a batch of image frames, so anything that outputs a batch works: VHS_LoadVideo in the example, or a Load Images folder batch if you prefer stills.
• use_ensemble - boolean, default off. Same trade as the image node: better accuracy at roughly double the per-frame cost. On a video, "double the per-frame cost" compounds fast, so think before enabling it for a long clip.

The output

• images - the annotated IMAGE batch, same frame count as the input. Feed it into VHS_VideoCombine to write a video file, or into a preview node to scrub through the results.

How to install it

It's part of the ComfyUI-Moondream-Gaze-Detection pack:

• ComfyUI Manager - search "Moondream", install, restart.
• Manual - from ComfyUI/custom_nodes/:

  git clone https://github.com/jhj0517/ComfyUI-Moondream-Gaze-Detection
  cd ComfyUI-Moondream-Gaze-Detection
  pip install -r requirements.txt

The requirements add matplotlib, pyvips, accelerate, and opencv-python. As with the rest of the pack, pyvips is the one that needs babysitting - Windows users get the native DLLs auto-downloaded, Linux users usually need system libvips installed. The Moondream 2 model downloads itself to ComfyUI/models/moondream/ on first load, and the pack runs at around 6GB peak VRAM.

Where people get burned

The big one is time. This is not a realtime node: every frame runs a full vision-model encode plus face and gaze detection, so a 10-second clip at 30fps is 300 independent inferences. With use_ensemble on, double that. Budget for a video this node to take minutes, not seconds, and test on a short crop first before committing a long clip.

Second is the flicker. Because each frame is re-analyzed independently, you'll see the arrows jitter between frames even on footage that looks static. The answer is not to tune a setting - there isn't one - but to accept it as a limitation of the per-frame approach, or add temporal smoothing downstream with another node.

Third is the accuracy ceiling, which the pack's author has been upfront about since he announced it: gaze detection on a small 2B model is solid for broad direction and rough on subtle eye contact. For video that compounds - a glance that lasts two frames may simply never register. Use it for "is the subject looking left or right," not for eye-contact forensics, and you'll be happy with it. It's cheap, fully local, and it puts moving arrows on faces; for most people that's exactly the toy they came for.