LUNA VLM Loader (GGUF)

The model you're running isn't the problem - its encoder is

Z-Image is the 6B model that put near-Flux quality on a 12GB card. But the text encoder it ships with, Qwen3-4B, eats about 8GB of VRAM in bf16 - bigger than the diffusion model itself. On a 10GB card, the encoder is the thing that decides whether you can run Z-Image at all. This pack fixes exactly that, and the LUNA VLM Loader is where the fix starts.

This node loads a GGUF-quantized Qwen3-VL into VRAM as Z-Image's text encoder, plus an optional vision projector, and hands you one LLM_MODEL object the other two nodes in the pack consume. Everything auto-downloads from HuggingFace on first use, so you never hunt for weights.

How it actually works

The trick is that it can't use the stock llama-cpp-python. Z-Image conditions on the penultimate hidden states of its encoder, not the final layer, and exposing those requires a custom fork of llama.cpp. The loader uses one (the LUNA fork), loads the GGUF with embeddings=True and per-token pooling, and then does the genuinely clever bit: it parses the quant tag out of the model filename (Q4_K_M, IQ2_XXS, …) and auto-downloads and loads the matching per-quant adapter - a small residual MLP (~160MB) trained to realign Qwen3-VL's hidden states to the distribution Z-Image's denoiser expects. You never select it; it just matches and loads. That's why even an IQ1_S quant, at roughly one bit per weight, still conditions cleanly.

The inputs that matter

• model_path - dropdown of the GGUF files (auto-downloaded if missing). The README recommends LUNA-Qwen3-VL.i1-Q4_K_M.gguf (2.4GB); Q3_K_S and Q2_K_S are the 6GB-card picks.
• n_gpu_layers - default -1, offload everything to the GPU. Leave it.
• n_ctx - context window in tokens, default 2048. Only raise it if you're encoding very long prompts.
• gpu_index - which CUDA device, if you're multi-GPU. Default 0.
• mmproj_path - set this to a projector (LUNA-Qwen3-VL.mmproj-Q4_K_M.gguf, 268MB) only if you plan to use the LUNA VLM Chat node. For pure conditioning, leave it at none and save the VRAM.

The output is llm_model, which wires straight into LUNA Text Conditioner (for conditioning) or LUNA VLM Chat (for image-to-prompt).

What it costs you

The README's numbers, in VRAM including the adapter:

| Config | VRAM | |---|---| | IQ1_S + adapter (conditioning only) | ~1.3 GB | | IQ2_XXS + adapter + Q4 mmproj | ~1.8 GB | | Q2_K_S + adapter + Q4 mmproj | ~2.1 GB | | Q4_K_M + adapter + Q8 mmproj (recommended) | ~3.2 GB | | Qwen3-4B bf16 safetensors (for comparison) | ~8.2 GB |

That's the 3.3× reduction the pack brags about, and it's real. The approach behind this pack showed up in a March 2026 r/StableDiffusion post measuring ~2.5GB total at 0.979 cosine similarity to the full-precision encoder - same fork-llama.cpp, train-an-adapter recipe, packaged here as nodes.

Installing it

Via ComfyUI Manager, search LUNA Z-Image Qwen3-VL. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/LSDJesus/LUNA-Z-Image-Qwen3-VL.git
cd LUNA-Z-Image-Qwen3-VL
pip install -r requirements.txt

Then restart ComfyUI. Model files land in ComfyUI/models/LLM/LUNA-Qwen3-VL/ and download themselves on the first run - you need internet the first time.

Where people get burned

• The stock llama-cpp-python will not work. requirements.txt pins a wheel from the LUNA fork, and it's mandatory - the penultimate-layer API doesn't exist in the official build, so you'll get a NULL-pointer error or worse. If another node pack already pulled in the official llama-cpp-python, this install replaces it (same package name). Usually fine; occasionally a node that wanted the stock API complains.
• There's no macOS wheel - only Windows CUDA and Linux (manylinux) wheels are listed. Mac users are compiling the fork from source.
• IQ1_S is conditioning-only. The README says it outright: VLM chat at ~1 bit/weight produces gibberish. Use it for encoding, not vision.
• First run downloads ~2.5GB from HuggingFace, so don't panic when nothing's in the folder before the first queue.