Gaze Detection

The node's display name tells you exactly what it does, which is rarer in ComfyUI than it should be. Gaze Detection takes an image, finds every face in it, and draws a colored bounding box around each one plus a fading arrow showing where the eyes are pointing. It's the "put laser eyes on it" node - the one people reach for when a photo or a rendered frame needs to answer the question "where is everyone looking?"

The bigger thing it represents: this is a ComfyUI wrapper around Moondream's gaze detection, a feature of the open-weights Moondream 2 vision-language model. That means no cloud API, no key, nothing leaves your machine. If you've seen Moondream mentioned around the ecosystem - it's a small vision model that also shows up in other people's packs - this is a very narrow slice of what it can do, applied to one task.

How it works

The pipeline is short and honest. The model encodes your image, runs a face detection pass, then for each face runs Moondream's detect_gaze. Each detected face gets a bounding box (positions normalized to image coordinates, so it works at any resolution), and the gaze output is a point in the same space. The node draws the whole thing with matplotlib: a rainbow-colored box per face, and a gaze line that fades out along its length toward a marker dot at the focal point. The annotated figure is rendered straight back into an image tensor.

It prints nothing useful to the console, by the way - the processing status is computed and thrown away. You judge the result by looking at the output image, which is how it should be.

Inputs that matter

Three inputs, and only one of them will occupy you:

• model - the MOONDREAM_MODEL from the pack's (Down)Load Moondream Model node. Non-negotiable; the pack won't run without the loader node wired in.
• image - any IMAGE tensor, so it takes the output of Load Image, or anything upstream like a VAE-decoded render or a ControlNet preview. Feed it what you want analyzed.
• use_ensemble - a boolean, default off. When on, the model also encodes a horizontally flipped copy of the image and uses it as extra signal during gaze estimation. It measurably improves accuracy and roughly doubles the inference cost. Set it when you care about the answer; leave it off when you're iterating.

The output

• image - the annotated IMAGE, same size as the input with boxes and gaze arrows drawn on. Wire it into Preview Image to look at it, or Save Image if you want the result on disk. You can also feed it onward through the graph if you want to crop around faces or feed it back into something - though for most people this is a terminal node.

How to install it

Install the ComfyUI-Moondream-Gaze-Detection pack, which is what ships this node:

• ComfyUI Manager - search "Moondream", install, restart.
• Manual - from ComfyUI/custom_nodes/:

  git clone https://github.com/jhj0517/ComfyUI-Moondream-Gaze-Detection
  cd ComfyUI-Moondream-Gaze-Detection
  pip install -r requirements.txt

It pulls in matplotlib, pyvips, accelerate, and opencv-python. The pyvips dependency is the one that bites: on Windows the pack auto-downloads the native libvips DLLs it needs, but on Linux you typically have to install libvips at the system level first. The model itself downloads automatically to ComfyUI/models/moondream/ on first load - peak VRAM is around 6GB.

The honest part: accuracy

Set expectations now, because the author of this pack did, publicly. When he announced it on r/StableDiffusion he said outright that it wasn't as accurate as he'd hoped - it misses subtle up-and-down glances and the community immediately started testing it on the classic "is he looking at her boobs or her face" sample. The consensus: solid for broad left/right gaze direction, genuinely rough on fine eye contact and quick glances. It's a small model doing a hard thing.

Where it shines: it's cheap, it runs entirely local, and 6GB of VRAM gets you there. use_ensemble on helps the finer cases. Treat the output as a directional hint - great for "is anyone looking at this spot in my render" - and don't build a proctoring startup on the first result you see. Start with use_ensemble off to check it runs, then flip it on for the real pass.