Youtu-VL (GGUF)

The GGUF node is the same Youtu-VL 4B model, but run through llama.cpp instead of transformers - and it's the one you'll actually want on a consumer GPU, or a Mac. The pack's own default model is Q8_0, and the GGUF quality ladder backs that up: Q8 is essentially indistinguishable from fp16 at half the size. If Q8 fits, just use it; that's the whole pitch of this node.

Why the split matters

The standard node gives you better precision headroom and BitsAndBytes quantization, but it's the one that breaks on Apple Silicon. This GGUF path runs through llama.cpp, which is the battle-tested, cross-platform route - install the metal build on macOS and it works, which the standard transformers node genuinely can't claim right now. It's also just faster on modest hardware: llama.cpp does CPU/GPU hybrid inference, so a 6GB card that chokes on the fp16 transformers build can chew through the Q8 GGUF with layers split across CPU and GPU.

How it works

The node downloads the GGUF weights plus a separate vision projector (mmproj-Youtu-VL-4b-Instruct-BF16.gguf) from tencent/Youtu-VL-4B-Instruct-GGUF into models/LLM/GGUF/Youtu-VL-4B-Instruct-GGUF/, then builds a llama_cpp.Llama instance with a Llava15ChatHandler so it can see images. Your ComfyUI IMAGE tensor is converted to a base64 PNG and sent through the chat-completion API. Q8 wants roughly 5.5GB of VRAM per the pack's config; the F16 file is ~10GB.

The inputs

Refreshingly few:

• model - Youtu-VL-4B-Instruct-GGUF-Q8 (default) or ...-GGUF-F16. Pick Q8 unless you have serious headroom; F16 buys you almost nothing measurable.
• preset_prompt - the six built-ins (Describe Image, Detailed Description, Generate Tags, OCR Text, ...).
• custom_prompt, max_tokens, keep_model_loaded, seed - standard. Optional image; output is a single text string.

Notably, the sampling is fixed internally at temperature 0.1 / top_p 0.001 - deterministic, which is what a captioner should be. n_gpu_layers is forced to -1 (all on GPU) and context to 4096. If you want to change either, you want the Advanced GGUF node.

Installation

llama-cpp-python is not in requirements.txt - it's an optional extra, and without it the node just raises an ImportError telling you the install command. The real one, per the README:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

Swap cu121 for your CUDA version (e.g. cu118) or metal on macOS. This is the #1 reason people report the GGUF node as "missing." Also note the README's troubleshooting mentions picking a Q4 or Q5 model - that's aspirational; the shipped config.json only offers Q8 and F16 in the menu. Q4/Q5 quants of Youtu-VL exist (mradermacher has repos), but you'd have to edit config.json to load them.

Common issues

Node missing or ImportError → install llama-cpp-python with the right wheel. OOM on a small card → this node has no VRAM knob; use the Advanced variant and drop n_gpu_layers. And the same licensing note as the rest of the pack: Youtu-VL's Tencent license is non-commercial and not intended for EU use.