DW Image2 Chat

DW Image2 Chat is the pack's pocket vision model: it runs moondream2 - a tiny vision-language model designed to run on a laptop - right on your machine, with no API key and no network calls. You feed it an IMAGE and a text_input question, and it answers in plain text. The display name is "DW Image2 Chat" and it was the pack's original image-backpropagation node (added July 2024), before the author layered on the bigger PaliGemma and Gemini vision options. It's the "good enough, cheap enough, private" choice.

The mechanism is straightforward: it saves your image tensor to a temp PNG, loads moondream2 with HuggingFace transformers, runs encode_image + answer_question with your text_input as the question, and returns the reply as a STRING. The model loads as a singleton (loaded once, reused across runs) but the node calls clear_memory() in a finally block - so it unloads the model from VRAM after every single run. That means every queue hit reloads moondream2 from disk, which is slow. First run especially: expect a wait.

Inputs: image (IMAGE) and text_input (multiline, your question - "describe this image", "what color is the hat?", "write a detailed caption"). Output: one STRING.

Installing it - model goes in an odd folder

cd ComfyUI/custom_nodes
git clone https://github.com/yiwangsimple/ComfyUI_GroqChat

Then manually download moondream2 (the README links huggingface.co/vikhyatk/moondream2) into this exact folder:

mkdir -p ComfyUI/models/LLavacheckpoints/files_for_moondream2
# put ALL the model files here

The LLavacheckpoints folder name (note the odd casing - LLava, capital C) is shared by the pack's other local models, and it's easy to mis-type. transformers, torch, and Pillow are in requirements.txt.

Honest take

The reply quality is on par with a small open vision model from 2024 - fine for captions and simple questions, not for subtle scene analysis. That's the trade for a model that runs on CPU if it has to. The reload-every-run behavior is the real annoyance; if you're batch-captioning hundreds of images, the Gemini Vision node (cloud, smarter) or the Ollama vision node (stays warm with keep_alive) will feel much faster. This one is best for occasional, private, offline image questions - and it's the most approachable local-vision intro in the pack.