CXH_SmolVlm_Load

CXH_SmolVlm_Load loads SmolVLM-Instruct from Hugging Face - a small vision-language model that fits on modest hardware, added to this pack as a lighter captioning/vision option alongside the big JoyCaption and MiniCPM setups. If you've ever wanted a vision model that doesn't demand 10 GB of VRAM, this is the pack's answer.

SmolVLM's pitch is efficiency: it's designed to run image+text tasks on a single consumer GPU without quantization gymnastics. The loader reads it in bfloat16 straight from the Hub into models/LLM/SmolVLM-Instruct (the pack's shared LLM folder again), then hands you a CXH_SmolVlm_Pipe for the run node.

What's on the node

Minimal, like most loaders here:

• Input: model - one choice, HuggingFaceTB/SmolVLM-Instruct.
• Output: pipe (CXH_SmolVlm_Pipe) - model + processor, loaded in bfloat16.

No precision or attention options. The loader uses AutoModelForVision2Seq, and the commented-out flash_attention_2 line in the source is a hint that it's deliberately staying on the default attention to avoid the flash-attn install headache.

Installing

Pack install - ComfyUI Manager (search "Comfyui_CXH_joy_caption") or:

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

Restart; the model downloads on first load.

Gotchas

• First download is a few GB into models/LLM/. The load is a normal from_pretrained, so be patient the first time.
• Device handling differs from the rest of the pack: this loader uses "cuda" if torch.cuda.is_available() else "cpu". It's the one loader in the pack that won't hard-crash without a GPU - but CPU inference on a VLM is slow enough that you don't want to test it.
• SmolVLM is small - as in, its captions are generally weaker than JoyCaption Alpha Two on complex scenes. Use it when VRAM is the constraint or the task is simple; reach for JoyCaption when caption quality is the point.
• The CXH_SmolVlm_Run node extracts the assistant's reply from the generation, so "Assistant:" noise is handled for you.