EmAySee_Vision LLM Local Generator

Caption → prompt pipelines are everywhere, but most of them phone home to an API. This node runs the whole thing locally: it loads a vision-language model on your own GPU, feeds it your image, and returns a generation prompt. No key, no cloud, no image leaving your machine. The name says "Qwen" but the code actually sniff-tests the model path - it'll use the right architecture for Qwen3-VL, Qwen2.5-VL, or Gemma VLM weights, and falls back to a generic class for anything else.

How it works

You give it an image and a model_path. On the first run it loads the model with the Hugging Face transformers stack, in bfloat16, with device_map="auto" so layers spill to CPU if the GPU fills up. Your image goes in as the user turn of a chat, the model generates a response, and the node splits out anything between <think> tags into a separate thinking output - nice for reasoning models that want to narrate before they answer. Then, if unload_after is on (it is by default), it frees the model immediately so your diffusion sampler gets the VRAM back.

The inputs that matter

• model_path - a local folder of HF-format weights. The default points at a /ai/datasets/... path from the author's own machine. It will not exist for you. Point this at your own Qwen3-VL or Gemma-3 VLM directory, or it errors on load.
• system_prompt - the default is a solid "generate a detailed image generation prompt" instruction. This is the whole game; tune it to get tag-style or prose-style output.
• user_prompt - extra instructions per run, can be empty.
• max_tokens (2048) and temperature (0.3) - keep temperature low for prompt extraction; crank it if you want variety.
• max_memory_percent (0.95) - how much of your VRAM and RAM the model may grab before it starts. Drop this if you're running a diffusion model alongside it.
• unload_after (true) - unloads the model after every call. Turn it off only if you're chaining several generations in one run.

The outputs

• prompt (STRING) - the cleaned-up generated prompt, ready to feed a CLIP encoder.
• thinking (STRING) - the reasoning text between <think> tags, empty if the model didn't reason.

Installing it

From the EmAySee_ComfyUI_CustomNodes pack, via ComfyUI Manager (search the pack name) or:

cd ComfyUI/custom_nodes
git clone https://github.com/EmAySee/ComfyUI_EmAySee_CustomNodes
# restart ComfyUI

This node is the exception to the pack's "no dependencies" rule: it needs transformers, torch, and PIL at minimum, and the model itself is several gigabytes of weights you must download yourself. The pack has no requirements.txt, so if transformers is missing you'll see a "transformers library is missing or out of date" error on the node - install it with pip install transformers in your ComfyUI environment first.

Gotchas

The load is slow and hungry. A 4B VLM will eat several GB of VRAM even in bf16; that's why unload_after defaults to true - leave it on or your next KSampler run will swap to system RAM and crawl. And double-check the path: absolute paths with no way to browse means one typo and you're staring at an import error. This is a genuinely useful node, but it's also the most machine-specific one in the pack.