LlamaVision Model

If you've ever wanted your ComfyUI graph to actually see the images it's making - caption a batch for a Flux dataset, describe a render, answer a question about a generated scene - this is the node that makes it possible. LlamaVision Model is the entry point of the YALLM-LlamaVision pack. It does exactly one job: pull Llama 3.2 Vision (the 11B Instruct model) into your workflow and hand you an llm_model to feed into its sibling node, LlamaVision Chat.

The name's a joke, by the way. "YALLM" is "Yet Another LLM," and the author, Allan Saddi, is upfront about scope: no API keys, no cloud calls, everything local. And no, he hasn't tested it with the 90B variant - this is an 11B pack, full stop.

How it works

Under the hood it's straight HuggingFace transformers. The node reads a models.yaml file (a models.yaml.default ships with the pack; copy it to models.yaml to customize), then downloads the chosen model with snapshot_download into ComfyUI/models/LLM - unless you set use_hf_cache: true in that yaml, in which case it lands in your usual HuggingFace cache. Load is MllamaForConditionalGeneration with device_map="auto" and SDPA attention, about as hands-off as an 11B multimodal load gets.

The inputs that matter

• model - a dropdown with three choices. The default is unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit, a pre-quantized NF4 model and honestly the one you want on most cards. meta-llama/Llama-3.2-11B-Vision-Instruct is the original BF16 - gated, so you need huggingface-cli login first, and it wants roughly 22GB of VRAM. SeanScripts/Llama-3.2-11B-Vision-Instruct-nf4 is a slightly different NF4 quantization the author never got around to testing.
• quantization - default, nf4, or int8. Here's the trap: this dropdown only really matters if you pick the unquantized meta-llama model, where it applies bitsandbytes on the fly. Pick a pre-quantized model and its own config overrides whatever you select.

What comes out: one output, llm_model, which wires straight into the llm_model input on LlamaVision Chat. Nothing else.

VRAM, honestly

The README's numbers match what I've seen: NF4 runs in about 10GB, which is the sweet spot. Int8 wants 15–15.5GB and - here's the nasty part - bitsandbytes cannot offload an int8 model, so it stays parked in VRAM forever. You realistically want 24GB before int8 makes sense, and the author will tell you the same. The BF16 original is a 20GB-every-load ordeal; just use a pre-quantized model. After a chat completes, NF4 is offloaded to system RAM automatically, so VRAM frees up between runs while the weights stay warm.

Installation

ComfyUI Manager: search "YALLM-LlamaVision", install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/asaddi/YALLM-LlamaVision
pip install -r YALLM-LlamaVision/requirements.txt

That pip line is the thing to watch. It wants transformers>=4.46, plus accelerate, bitsandbytes, and pydantic, and it will happily upgrade your existing transformers to get there. In a shared ComfyUI environment that can break other nodes that pinned older versions - the classic custom-node dependency hell. Keep an isolated venv if you run anything else LLM-adjacent.

Troubleshooting

• First run downloads the model - a multi-gigabyte snapshot. Nothing's wrong, it's just slow.
• The meta-llama model errors with a gating message → log in with huggingface-cli login.
• Out of memory → you picked int8 or BF16. Switch to the default NF4.
• Want a different model? Edit models.yaml: name, HF repo id, and whether to use the HF cache.

One last thing, because googling "LlamaVision" is how people get spooked: the 2024 "ComfyUI_LLMVISION" custom-node malware is a different, unrelated thing. This is a small, readable, BSD-licensed pack - and reading the source before you trust any node is a habit worth having in this ecosystem.