CXH_SmolVlm_Run

CXH_SmolVlm_Run is the inference half of the pack's SmolVLM option - give it a pipe from CXH_SmolVlm_Load, an image, and an instruction, and it returns the model's text reply. It's the low-VRAM captioning path in a pack whose other vision nodes want a big card, and it fills a real gap: JoyCaption Alpha Two needs ~10 GB just for the caption model, while SmolVLM was built to run on much less.

Worth being honest about the trade: on caption quality, SmolVLM generally trails JoyCaption. This node is the "I only have 8 GB and I still want local captions" choice, or the quick-and-dirty option when you're captioning simple images and don't want to babysit a giant download. The KB's captioning advice still points at JoyCaption/Florence 2 as the quality bar - SmolVLM is the budget seat.

How it works

Straightforward vision-language inference. The image is converted to PIL, bundled with your prompt into a chat message ("type": "image" + "type": "text"), passed through the processor's chat template, and generated with max_new_tokens. The output is decoded and then post-processed with a regex that grabs everything after the Assistant: marker - so you get the reply, not the whole prompt echo. If the regex doesn't match (sometimes it just doesn't), it returns the full generated text instead.

Inputs and output

• pipe - from CXH_SmolVlm_Load.
• image - any IMAGE.
• prompt - multiline instruction; default is the pack's standard "describe the image and produce a short SD prompt" text.
• max_tokens (10–4048, default 1024), seed (the source passes it but doesn't force deterministic sampling).
• Output: one STRING - the assistant's reply.

Installing

Same pack install - ComfyUI Manager (search "Comfyui_CXH_joy_caption") or:

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

Troubleshooting

• Output occasionally includes prompt echo - that's the regex fallback doing its thing; harmless but ugly in a Show Text node.
• Weak on complex scenes: if captions come back generic, that's the model's size, not your setup. For quality captions use the JoyCaption nodes.
• CPU fallback exists (unlike the rest of the pack) but you'll wait. A GPU of any stripe is the intended path.
• For a batch of images there's no dedicated batch node for SmolVLM - you'd loop the run node or use the JoyCaption batch nodes instead.