LUNA VLM Chat

The bonus feature that makes the pack feel like a cheat code

The LUNA VLM Loader loads Qwen3-VL as a text encoder - but it's a vision-language model. LUNA VLM Chat is where that pays off: feed it an image and a question, get descriptive text back, all from the same ~2.4GB weights you're already holding in VRAM for conditioning. No second model, no offload dance, no extra card. Just add a 268MB projector.

Its job is image-to-prompt. You drop in a reference image, it writes a detailed description, and you hand that text to the LUNA Text Conditioner (optionally concatenated with your own prompt) and sample. That's the classic img2img / style-capture loop - describe the reference, generate in the same direction - and it's the VLM-captioning role the community reached for with Qwen3-VL generally, here with zero extra VRAM cost.

How it works

Under the hood it uses llama.cpp's multimodal (mtmd) pipeline. The loader must have loaded a projector, then Chat builds a multimodal context from it, converts your ComfyUI IMAGE tensor to PNG bytes, tokenizes the chat with an image marker inline, decodes the image chunks into the context, and generates text. Sampling is handled in the node itself, with temperature and a repetition penalty applied to generated tokens - which is why the default repetition_penalty of 1.15 keeps it from falling into a loop.

If the model wasn't loaded with an mmproj, the node refuses with a clear error telling you to reload with a projector. It won't guess.

The inputs that matter

• llm_model - from the LUNA VLM Loader, and it must have an mmproj loaded.
• image - the reference image, any ComfyUI IMAGE output.
• prompt - what you want it to say. The default, "Describe this image in detail for use as an image generation prompt," is a fine starting point.

The knobs, all optional:

• max_tokens - 256 default, up to 2048. Raise it for long descriptions.
• temperature - 0.7 default; 0 is fully greedy/deterministic.
• repetition_penalty - 1.15 default; 1.0 turns it off. Bump toward 1.3 if you see looping.
• system_prompt - steer the persona ("describe only the clothing", "be terse", etc.).
• prepend_text / append_text - wrap the output. Handy for forcing a fixed prefix into the final prompt.

The output is a plain text STRING. Wire it into the LUNA Text Conditioner, or into a string-concat node to merge it with a manual prompt before conditioning.

Installing it

Same pack as the rest - install once, three nodes. ComfyUI Manager (search LUNA Z-Image Qwen3-VL) or:

cd ComfyUI/custom_nodes
git clone https://github.com/LSDJesus/LUNA-Z-Image-Qwen3-VL.git
cd LUNA-Z-Image-Qwen3-VL
pip install -r requirements.txt

Then restart. The requirements pin the LUNA fork of llama-cpp-python (the stock one won't work - the mtmd API it calls isn't in the official build), and there's no macOS wheel, so Mac users compile the fork.

Where people get burned

• Forgetting the mmproj is the #1 mistake. The loader defaults to none; if you pick the model but never set the projector, Chat throws the "requires an mmproj" error. Reload with LUNA-Qwen3-VL.mmproj-Q4_K_M.gguf selected and it just works.
• IQ1_S chat is gibberish. At ~1 bit/weight the model can still condition, but it can't talk. Use Q2_K_S or higher for chat.
• Multi-subject attribution is every VLM's weak spot. With two people in frame, expect mix-ups over who's wearing what. For prompt-seeding that's cosmetic; if you're captioning a training dataset, audit the output by hand.
• Don't chain this into conditioning and expect the negative-prompt semantics to survive - the conditioning path is an LLM encoder, so CLIP-era tricks like prompt weighting and negative embeddings don't apply downstream.