Youtu-VL (GGUF Advanced)

If AILab_YoutuVL_GGUF is the "it just works" llama.cpp node, this is the one that lets you decide how much of the model lives on your GPU and how chatty it gets. Same Q8/F16 Youtu-VL weights, same Llava15ChatHandler vision path, same single text output - but two knobs the plain GGUF node hardcodes, plus full sampling control. It's the GGUF node I'd actually put in a workflow on a 6–8GB card.

The knob that matters most

n_gpu_layers. The plain GGUF node forces -1, meaning everything offloaded to the GPU. Here, -1 still means all layers on GPU, but you can drop it to, say, 20 and let the rest run on CPU. That's the difference between "OOM on my 6GB card" and "slow but it runs." It's the llama.cpp equivalent of the VRAM tap that most quantization tools give you, and for a 4B model it's genuinely usable - you're not offloading a 70B dinosaur.

Second is n_ctx, the context window, from 512 up to 32768 (default 4096). The practical effect is small for single-turn captioning - a caption doesn't need a big context - but it matters for longer visual QA exchanges or if you push max_tokens high. Each extra token of context costs RAM, so leave it at 4096 unless you have a reason.

Sampling controls

The trio the base GGUF node hides:

• temperature - default 0.1; 0.1–0.4 is focused, 0.7+ creative. The base node pins 0.1 and won't budge.
• top_p - default 0.001, near-greedy. Loosen it along with temperature if you want varied answers.
• max_tokens - raised to 32768 here.

Everything else matches the plain GGUF node: model (Q8 default, F16), the six preset_prompt options, custom_prompt override, keep_model_loaded, seed, optional image, and a text output to wire into a Show Text node or a prompt-builder.

Installation

Same pack, same extra step - llama-cpp-python is not in requirements.txt, so:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

...with cu118 or metal swapped in to match your system. This node is also the Mac-friendly route for the pack, since the standard transformers nodes hit a documented 'sdpa' crash on Apple Silicon while the llama.cpp path runs fine.

Gotchas

If the model won't load or the node errors at import, it's the llama-cpp-python wheel nine times out of ten - the wrong CUDA suffix installs fine and then fails at runtime. And don't let the README's mention of Q4/Q5 models lead you astray: the shipped model menu only lists Q8 and F16; the lower quants exist in the community but aren't in config.json. Remember the license too - Youtu-VL is non-commercial and not intended for EU use.