Moondream 3 / 3.1 Query

Caption nodes tell you what a model thinks the image is. Query lets you ask it a question instead: "is the mug on the left or right of the keyboard", "what color are the shoes", "are there people in this frame". It's the VQA workhorse of the Moondream 3.1 family, and since the pack keeps the whole runtime isolated, you get real visual question-answering locally with no API key and no cloud round-trip.

Mechanically it's simple: a model handle from the Loader, a question (multiline, default "Describe this image precisely."), an optional image (or image_index to pick a frame from a batch), and it returns the answer as a STRING. What makes it worth choosing over just shoving the image through a text model is that Moondream 3.1 is actually good at spatial questions - "which side", "how many", "what's in front of" - which generic caption models routinely muff.

The interesting control is reasoning. It's off by default, which is the right call for most use: reasoning tokens cost time and you don't need them for "what color is the hat". Flip it on when you want the model to walk through its logic before answering, and the reasoning trace comes back in the separate reasoning_json output - useful when an answer looks wrong and you want to see why. max_tokens (512 default) caps the response, and stream_output will stream tokens into a connected View Text node so you can watch it work. unload_after=false keeps the shared worker resident between queries, which is the difference between "feels snappy" and "reloads a model every question."

Outputs: answer (the string you wire downstream), reasoning_json (empty when reasoning is off), and performance_json with the worker's FPS/timing report.

Installing it

Pack first - ComfyUI Manager (search "ComfyUI VLM nodes") or clone + requirements.txt. Then the Moondream isolated runtime, exactly as in the Loader article:

runtime="ComfyUI/models/LLavacheckpoints/moondream31-runtime"
uv venv "$runtime/.venv" --python 3.12
uv pip install --python "$runtime/.venv/bin/python" \
  -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements-moondream31.txt

Accept the Moondream Model License 1.0 on the loader, let the model download on first run. NVIDIA Ampere-or-newer or Apple Silicon only for local Photon - this node has no CPU/ROCm path.

Common issues

• Answer ignores the video: if you connect both an image and a video batch, the video wins - the still image socket is ignored for video inference so the model can't silently answer from the wrong media. Remove the image when working with video frames.
• Wrong frame answered → image_index selects which batch frame gets queried; the default 0 might not be the frame you care about.
• Garbage spatial answers → try reasoning=true. It's slower, but for "is it on the left or the right" style questions the reasoning trace is frequently the difference between a guess and a correct answer.
• Question too vague → Moondream rewards specificity. "Describe the scene" is a captioning task; "how many chairs are visible" is a query task. Frame questions like the latter and you'll be happy with the results.

This node slots into workflows wherever a conditional needs to read an image - content moderation gates, scene checks before a video pipeline, auto-tagging with a verdict prompt. It's the "ask the image" node, and for local hardware it's about as good as it gets right now.