MinusZone - ModelConfigManualSelect(ImageInterrogator)

ModelConfigManualSelect(ImageInterrogator) is how you use your own vision model files with this pack's image interrogator, instead of the downloader's preset list. It scans ComfyUI/models/gguf for GGUF files, you pick a base LLaVA/vision model and a matching mmproj projector, and it hands the pair to CLIPTextEncode(ImageInterrogator).

Why bother when the downloader exists? Because the preset list is fixed, and the world of vision GGUF is bigger than it. You may already own a ggml_llava-v1.5-7b, a MiniCPM-V, or a quant you prefer. Or you may be following the golden rule that Q4 vision models describe images worse than Q5/Q6 - the manual node lets you point at whatever quant you downloaded yourself.

The three inputs

• llama_cpp_model - every .gguf under ComfyUI/models/gguf (relative paths, subdirectories included). Pick your vision base model.
• mmproj_model - auto plus the same GGUF list. Leave it on auto and the node tries to match a projector to your base model automatically by hashing the file and looking it up in the pack's model_zoo; if that fails it errors rather than guessing. Set it explicitly when you have a known-good pair.
• chat_format - auto + llama.cpp chat handlers. auto for LLaVA's standard templates.

Output is an image_interrogator_model config for the image_interrogator_model input of CLIPTextEncode(ImageInterrogator).

How it works

At encode time the node joins your chosen filenames onto the gguf path, resolves the projector (auto-match by SHA256 against the model zoo, or your explicit pick), and loads both through llama.cpp - the mmproj file is what lets a text LLM "see" images. This is the same two-file dance the downloader automates, just with your files and no download step. If either file is missing, you get an error instead of a fetch.

Installing

Part of ComfyUI-Prompt-MZ:

cd ComfyUI/custom_nodes
git clone https://github.com/MinusZoneAI/ComfyUI-Prompt-MZ

Restart. Requires llama-cpp-python + diskcache. If you want a LLaVA pair and have none, the downloader variant (ModelConfigDownloaderSelect(ImageInterrogator)) will fetch one for you; this node is for when you already have the files.

Troubleshooting

The Failed to automatically find the corresponding mmproj file error means auto-match couldn't pair your base model to a known projector - normal for models not in the zoo. Fix it by picking the mmproj explicitly from the dropdown; you need the correct pair for the exact model family (e.g. a LLaVA-1.6 Mistral mmproj won't work with a Llama-3 base). Second trap: picking a non-vision text model as the base - it'll load, but the interrogator will produce nonsense or boilerplate instead of image descriptions; there's no validation that you've selected a vision model. And the usual pack FAQ items apply: stale llama-cpp-python (LLAMA_SPLIT_MODE_LAYER) and CUDA mismatch (LLama.dll) are the two load failures to expect, both fixed by upgrading the pip package / switching to CUDA 12.1 PyTorch.