MinusZone - ModelConfigDownloaderSelect(ImageInterrogator)

Image interrogation - "read this image back as a text prompt" - needs a vision-language model, and in llama.cpp that means two files: a base LLM and a projector (mmproj) model that maps image embeddings into text space. ModelConfigDownloaderSelect(ImageInterrogator) is the download-it-for-you version: pick a model name, let it fetch the base GGUF, and it even auto-matches the right mmproj by SHA256. Feed the resulting config into CLIPTextEncode(ImageInterrogator) and you're reading images.

The curated list is genuinely useful because vision models are where llama.cpp file pairing gets fiddly. It offers llava-phi-3-mini-int4, llava-llama-3-8b-v1_1-int4, ggml_llava1_5-7b-q4_k_m, ggml_bakllava-1-q4_k_m, and llava_v1_6_mistral_7b_q5_k_m - a solid spread from tiny int4 (fast, low VRAM) to the Mistral-7B Q5 for better descriptions. The default recommendation for most cards is the LLaVA 1.6 Mistral Q5.

The three inputs

• model_name - the base vision model. First run downloads it into ComfyUI/models/gguf from ModelScope, checksum-verified.
• mmproj_model_name - defaults to auto, and this is the clever bit: the node SHA256-hashes the base model you picked, looks the hash up in the pack's model_zoo.json, and finds the matching projector automatically. Manually choosing among auto + the mmproj presets is only needed when auto-match fails.
• chat_format - auto plus llama.cpp's chat handlers. auto handles detection for the standard LLaVA templates.

Output: an image_interrogator_model config feeding the image_interrogator_model input on CLIPTextEncode(ImageInterrogator).

How it works

Same pattern as the LLM downloader: this node returns a "DownloaderSelect" config; the actual download happens on first interrogation run. Auto-matching the mmproj by SHA256 is the standout mechanism - the model_zoo maps each base model's file hash to its matching projector, which spares you the "which mmproj goes with which model" guessing game that trips up everyone doing LLaVA-in-llama.cpp by hand.

Installing

Part of ComfyUI-Prompt-MZ - ComfyUI Manager, search "ComfyUI-Prompt-MZ", or:

cd ComfyUI/custom_nodes
git clone https://github.com/MinusZoneAI/ComfyUI-Prompt-MZ

Requires llama-cpp-python + diskcache. Note the README FAQ items apply hardest here: vision models through llama.cpp are exactly where stale versions and CUDA mismatch bite (LLAMA_SPLIT_MODE_LAYER, LLama.dll). Upgrade llama-cpp-python, run PyTorch on CUDA 12.1, and if the wheel build fails, it's a network problem - proxy up or install the package manually.

Troubleshooting

The classic failure is Failed to automatically find the corresponding mmproj file - auto-match came up empty, usually because your base model file was swapped or renamed (the hash no longer matches the zoo). Fix: pick the mmproj manually from the dropdown, or re-download the base model so its SHA256 matches. Second: after the first big download, the pack keeps models resident when keep_device is on; a memory spike mid-batch is normal, and keep_device off forces a reload per image, which is slower but kinder to VRAM. And if the captions come back as boilerplate or empty, the retry logic in ImageCaptionerConfig handles it - but the vision model itself being Q4-weak is often the real reason descriptions are thin.