Moondream 3 / 3.1 Detect (Image / Video)

Detection is where Moondream 3.1 gets properly interesting, and this node is the one that does it justice. Detect runs open-vocabulary detection - you type what you're looking for ("person", "mug", "red car", "safety vest") and it draws bounding boxes around every instance - on a still image or a full video frame batch. The word "open-vocabulary" is the key: you're not limited to a fixed class list like a YOLO model. Ask for anything, get boxes.

The mechanism is what makes this node feel engineered rather than bolted on. Moondream runs in the isolated Photon worker, and detection submits frames concurrently so Photon can form dynamic GPU batches - that's what parallel_requests (default 4) controls, and it's the difference between processing video at a crawl and actually keeping up. frame_stride (1 = every frame, 2 = every other frame) trims the workload when you don't need per-frame detection, fps just tells the node the video's rate so timestamps come out right, and max_objects caps how many instances it'll report per frame.

The object field is a single-line prompt for what to find - "person" by default, and it's worth being specific. And here's a detail that shows the pack's care: every run reports measured worker FPS, end-to-end FPS, and real-time factor in performance_json, so you can actually see whether your settings are keeping up with real-time. That's rare and genuinely useful.

Outputs are where this node pays off. detections is the pack's typed VLM_DETECTIONS socket - per-frame boxes, labels, scores, and coordinates in source-image pixels - and it plugs straight into the pack's detection utilities (Filter, Select, Crop, Render, Detections-to-Masks/BoundingBoxes). Alongside it you get detections_json (the same data as text, for API/save), preview (an IMAGE with boxes drawn), box_masks (a MASK per detection), and both bounding_boxes (core Comfy BOUNDING_BOX) and bounding_boxes_with_metadata (BOUNDING_BOXES). You can wire the boxes into core Comfy workflows without ever touching the JSON.

Installing it

The pack install is standard - ComfyUI Manager (search "ComfyUI VLM nodes") or clone + requirements.txt - plus the Moondream isolated runtime:

runtime="ComfyUI/models/LLavacheckpoints/moondream31-runtime"
uv venv "$runtime/.venv" --python 3.12
uv pip install --python "$runtime/.venv/bin/python" \
  -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements-moondream31.txt

Accept the license on the Loader node (license_accepted), which is also where you set max_batch_size - the loader's 4 is a strong starting point for video. NVIDIA Ampere-or-newer or Apple Silicon for the local runtime.

Common issues

• Detection slows to a crawl on long clips → you're likely sending every frame. Raise frame_stride and, if you have the VRAM headroom, parallel_requests so Photon can batch better.
• Misses objects → the box you get is what the model's pointing skill settled on; try a more specific object string ("person in a yellow jacket" beats "person" for hard cases).
• Boxes off by a frame or two → pass the correct fps for the source video so timestamps line up with real time.
• Video and still image both connected → the video batch wins; the image input is ignored for video inference.
• Performance JSON looks bad → that's the point of the report - if end-to-end FPS is far below worker FPS, the bottleneck is frame feeding, not the model. Trim and downscale the batch before the node (the pack's VLM Image Pixel Budget node exists for exactly this).

For a single image, detection is instant and satisfying. For video, treat it as a batch job you tune with stride and parallelism - that's the workflow the node was actually built for.