VRS Native Vision Loader (Gemma FP16)

One dropdown, one output, and a completely reasonable question: why does this need its own loader when the pack's prompt nodes can have a Gemma model pushed into them?

Because retrieval and generation want different halves of the same model. This node loads only the vision tower - no language weights, no tokenizer, no generation - and hands it to the nodes that turn images into vectors.

What it's for

The pack (RS-Seasonal-Prompt-Generator) has a local reference-memory system: encode your own images, store vectors plus tags, query by new image. VRS Native Build and VRS Native Recall both take a VRS_ENCODER, and this is the only node that produces one.

Fifteen gigabytes of Gemma would be a ridiculous price for "tell me what's in this picture, in vector form". So the loader is a scalpel: it opens the safetensors file, keeps only tensors whose keys start with vision_model., converts them to FP16, and instantiates ComfyUI's Gemma 4 vision encoder with them. Resident weight footprint is roughly 319 MiB. Everything else in the file stays on disk.

That's also why it's separate from the prompt expander. The pack says it explicitly: the two loaders do not share vision tensors, and the language model in the expander is additional memory on top. If you're running a recall → expander graph, budget for both.

How it works, precisely

The model_name dropdown is populated straight from ComfyUI's text_encoders folder listing. Whatever .safetensors sits in ComfyUI/models/text_encoders/ shows up here. With nothing there, the list is empty - which is the most common "broken node" report for this one, and it's not broken, you just haven't put the file in.

On load it validates rather than assumes:

• The path must end in .safetensors. GGUF and mmproj are rejected outright - this is the native path, and the TIPO Q4 files are a different world.
• Tensors must be unquantized floating point. Vision weights get cast to FP16 for storage; computation runs through ComfyUI's Gemma 4 path in FP32 manual_cast.
• If there are no vision_model.* keys, you get No native vision_model weights; use the converted Gemma 4 E2B file - i.e. that's a full-text or huggingface-native layout, not the converted one this expects.
• CUDA is required. No CPU fallback: VRS native currently requires CUDA; CPU fallback is disabled. AMD and Apple users, that's your answer before you spend an hour on it.

It then computes an encoder contract: SHA-256 over every FP16 tensor plus the ComfyUI gemma4 module source, the torch and torchvision versions, and the preprocessing/pooling identifiers. That hash is what gets compared against a memory's manifest later. It's why swapping the model file or upgrading torch can force a rebuild - vectors from different contracts simply aren't comparable, and the pack refuses to pretend otherwise. The upside: Incompatible VRS encoder becomes a clear message instead of mysteriously mediocre retrieval.

Output is a single typed VRS_ENCODER. Wire it to VRS Native Build when writing, and to VRS Native Recall when reading - and use the same file in both, forever, for a given memory.

Where the file comes from

The pack does not ship it, and the VRS panel won't download it. You need a native, unquantized Gemma 4 E2B safetensors with vision tensors inside. If you already converted a full model for the native prompt nodes and put it in models/text_encoders/, try that first - the loader only reads the vision prefix, so a full file works.

If you don't have one, the repo includes a preparation script that pulls only the vision slice of the Heretic ARA release using HTTP range requests instead of downloading ~10 GB of language weights:

cd ComfyUI/custom_nodes/RS-Seasonal-Prompt-Generator
python -m vrs_int.prepare_encoder /some/scratch/dir

It writes model.safetensors plus config.json, preprocessor_config.json and a provenance.json receipt (model revision, processor revision, tensor count, byte count, per-file SHA-256) into that directory. Copy the safetensors into ComfyUI/models/text_encoders/ and restart. No credentials, no remote Python code, tensors unchanged - which is the kind of thing I like to see spelled out.

The rest of the install is the standard one:

cd ComfyUI/custom_nodes
git clone https://github.com/raspie10032/RS-Seasonal-Prompt-Generator

No pip packages for this path - it rides on ComfyUI's own PyTorch, safetensors and model management. No llama.cpp, no Transformers, no bitsandbytes, and do not replace torch.

Where it bites

Empty dropdown after adding the file. ComfyUI caches text_encoders listings. Restart the server, not just the browser tab.

You pointed it at a Q4 GGUF because that's what the prompt nodes use. Different file, different purpose. The Q4 path is for language generation; VRS needs the native safetensors.

Rebuild after an upgrade. Change the ComfyUI Gemma implementation or your torch version and the contract changes. Existing memories then look incompatible. That's not the pack being fussy - it's the pack refusing to give you plausible-looking garbage - but it does mean you should note your versions alongside any store you plan to keep for a while.

VRAM isn't zero just because it's 319 MiB. Resident weights are small; activations, the image preprocess and the search blocks all add up, and a full recall is a second GPU consumer alongside whatever diffusion model you're running. The author measured about 425 MiB peak PyTorch allocation for a single-image vision pass on a 4070 Ti SUPER - useful as a floor, not as a promise.