📦 AITEC LLM Vision Loader

AITEC LLM Vision Loader is the sibling of AITEC LLM Loader with one extra dropdown: an mmproj_file. That mmproj - the multimodal projection file - is what turns a text-only GGUF into something that can actually see an image, and it's the entire reason this node exists. You feed it a model file plus a matching mmproj-*.gguf, and it hands a vision-capable model pin to the AITEC LLM Vision node.

Why it's fiddlier than the plain loader: the mmproj has to match its model family, and the code works by hunting through a hardcoded list of vision chat handlers in llama_cpp.llama_chat_format - Gemma4, Gemma3, Qwen3, Qwen2.5-VL, GLM-4.x-V, MiniCPM, Llava 1.5/1.6, Moondream, and more. It tries each handler in order and uses the first one that exists in your llama-cpp-python build. If none of them are found, you get a clear error telling you the problem: your llama-cpp-python build isn't vision-capable, and you need the JamePeng fork. That's not optional advice - the stock pip install llama-cpp-python often ships without these handlers, and this is the single most common reason the vision nodes fail.

Same model location and inputs as its sibling, plus one:

• model_file - the main GGUF, from ComfyUI/models/llm/.
• mmproj_file - the matching projection file, also from ComfyUI/models/llm/. Pick the file that belongs to your model family; a Qwen mmproj won't make a Gemma model see.
• n_ctx - context, default 4096. Vision conversations plus thinking blocks eat context fast; the README's 16384+ guidance for reasoning models applies here harder.
• n_gpu_layers - -1 offloads everything.

Output is a single model (LLM_MODEL), consumed by AITEC LLM Vision. Like the text loader, it caches by filename + settings, so multiple Vision nodes share one loaded copy.

The install story is the pack-wide one with the extra manual step front and center:

cd ComfyUI/custom_nodes
git clone https://github.com/AI-TEC/ComfyUI-AITECCAFE-Toolkit
pip install -r ComfyUI-AITECCAFE-Toolkit/requirements.txt
# vision REQUIRES a vision-capable llama-cpp-python:
# grab the matching .whl from JamePeng's releases

And budget your VRAM honestly. The README's warning is blunt: "LLM Vision is heavier than regular LLMs" - you're loading the base model and the projection model on top of whatever generation models your workflow already holds. On a consumer card, a vision LLM next to a checkpoint is a squeeze; unload_after_run on the Vision node (on by default there) is how you survive it.

Honest take: this is the right abstraction - model plus projection as two dropdowns is exactly how you'd want it. The pain is purely in the environment: getting a vision-capable llama-cpp-python installed and finding matching GGUF + mmproj pairs. Once that's true, this loader is the easy half of a local image-captioning pipeline.