Youtu-VL

This is the node that runs Tencent's Youtu-VL 4B vision-language model inside ComfyUI - fully local, fully offline once the weights are downloaded, no API key, no cloud. You feed it an image and it hands you back text: a caption, a set of Danbooru-style tags, the visible text in a screenshot, or an answer to whatever you typed. It's the node in this pack you'll actually reach for.

Why you'd reach for it

Because a surprising amount of ComfyUI work starts with understanding an image before generating one. Turning a reference photo into a usable Stable Diffusion prompt, auto-captioning a dataset before training a LoRA, pulling OCR out of a UI mockup - that's all "image in, text out," and this node does it in one step. The captioning context matters here: for training captions the community's favorite is JoyCaption, and anime bases want WD14 tags - but this is a solid general-purpose option when you want captions and OCR and visual QA from the same 4B model, in a workflow you already have open.

How it works

Under the hood it's the standard transformers path: AutoModelForCausalLM plus AutoProcessor with trust_remote_code, built on Youtu-VL's SigLIP2 vision encoder. On first run it downloads ~25 files to models/LLM/Youtu-VL (about 8GB) - the "zero-config auto-download" is real, but the first run is not instant. Quantization uses BitsAndBytes: 4-bit NF4, or 8-bit that deliberately skips the vision tower and lm_head modules to dodge the LayerNorm/lm_head.SCB load errors bitsandbytes throws at them. There's also a neat safety valve: the node checks free VRAM against config.json's requirements and silently steps you down FP16 → 8-bit → 4-bit if you're about to OOM.

The inputs that matter

• quantization - the one you'll set. FP16 is best quality and wants ~8GB; 8-bit suits 8–16GB cards; 4-bit fits 6GB or less.
• preset_prompt - six built-in instructions: Describe Image, Detailed Description, Analyze Elements, Generate Tags, OCR Text, Art Style Analysis. Each is a system prompt in config.json.
• custom_prompt - multiline free-text that replaces the preset when filled. This is how you ask "what color is the hat?"
• max_tokens, keep_model_loaded, seed - boring, just set and forget.

The only input you might leave empty is the optional image; without one it still answers text prompts. Output is a single text string - wire it to a Show Text / Preview Text node, or feed it into a prompt-builder for an img2img loop.

Installation

ComfyUI Manager, search "ComfyUI Youtu-VL" (publisher 1038lab), or:

cd ComfyUI/custom_nodes/
git clone https://github.com/1038lab/ComfyUI-Youtu-VL.git
cd ComfyUI-Youtu-VL
pip install -r requirements.txt

One real gotcha buried in requirements.txt: it needs transformers>=4.56.0, which is newer than what some stock ComfyUI environments ship. If the node loads but dies at import or load time, upgrade transformers first.

Common issues

The documented failure is Apple Silicon: in the pack's own release thread, a Mac user hits an 'sdpa' exception during processing even though the node reports "Node on apple_silicon." On macOS, skip this node and use the GGUF variant - that's the reliable path. Also note the licensing quirk: Youtu-VL's Tencent license is non-commercial and not intended for use in the EU. And don't go hunting for the segmentation/depth/pose nodes the README teases in Beta/ - they're not in the shipped node set yet.