Gemma3 Vision Loader

Gemma 3 isn't just a text model. The 22.7GB safetensors LTX-2 uses as its text encoder contains a full SigLIP vision tower - the part that lets the model "see" an image when one is attached to a chat turn. This node's entire job is to pull that vision tower out of the Gemma 3 file and hand it to ComfyUI as a normal CLIP_VISION model. One input, one output, no math of its own. It's the front half of a two-node experiment (the other half is LTX2AdvancedCondition), and it exists because LTX-2 is multimodal in the model but single-modal in your prompt box.

Here's the problem it solves. The stock LTX-2 workflow feeds Gemma 3 text and nothing else; there's no built-in path for the conditioning to read a picture the way Gemma 3 can when you attach an image to a chat prompt. The pack's conditioning node splices image-derived tokens into the text stream, and to do that it needs a vision encoder that speaks Gemma 3's dialect. Your SDXL CLIP won't cut it. So first you need this loader.

How it works

Point it at a file in models/text_encoders and it loads the whole thing, then builds a ClipVisionModel from the bundled SigLIP config (clip_vision_siglip_892.json: hidden size 1152, 27 layers, 14px patches, 896px input), and keeps only the keys that vision model actually consumes. The language-model weights get dropped on the floor. What comes out is a CLIP_VISION you feed into the stock CLIP Vision Encode node next to your image - that node turns it into a CLIP_VISION_OUTPUT, which is exactly the input LTX2AdvancedCondition wants.

One thing to note: this isn't an officially supported extraction path. ComfyUI's ClipVisionModel wasn't built to parse a multimodal checkpoint this way, and the author knows it - the README opens with "experimental" and gets more honest from there. More on that below.

The input that matters

There's exactly one: text_encoder, a file picker over your models/text_encoders folder. Use the same Gemma 3 12B safetensors you already downloaded for LTX-2 (often named gemma-3-12b-it.safetensors, fp16 or an fp8 repack). Two gotchas: it has to be a .safetensors, because the loader reads with ComfyUI's torch loader rather than llama.cpp - a GGUF quant won't load. And it's the full 22GB (or ~11GB fp8) file, since the SigLIP weights live inside the same archive as the text weights.

Output is a single CLIP_VISION.

Install

No extra dependencies, no requirements.txt, nothing to pip install. The real dependency is a recent ComfyUI - the code imports comfy.text_encoders.llama and the newer comfy_api schema, both of which need a current build.

cd ComfyUI/custom_nodes
git clone https://github.com/nistvan86/comfyui-inagy-ltxtools

Then restart ComfyUI. Or use ComfyUI Manager and search "comfyui-inagy-ltxtools". The model file (Gemma 3 12B text encoder) goes in ComfyUI/models/text_encoders/ - if you run LTX-2 at all, you already have it.

Common issues

• OOM on load. You're asking it to read a 22GB file just to get the vision tower. Gemma 3 12B is famously the biggest OOM source on LTX-2 setups; on a consumer card use an fp8 repack and expect a slow first load.
• NaN / "Conditioning tensor got lost" errors. Those come from the sibling node, but they're this pack's signature bug: the author documents Gemma3MultiModalProjector occasionally spitting out all-NaN tensors, fixed by clearing the model cache (or restarting).
• Node doesn't appear after install. Your ComfyUI is too old for the comfy.text_encoders.llama import. Update ComfyUI first.

The honest verdict, from the horse's mouth: the author says the pack "doesn't really seem to work correctly" - it does something, but the conditioning picks up artifacts and "it even affects the audio." This is a look-under-the-hood experiment for people curious how Gemma 3's multimodal conditioning ticks, not a production tool. Set your expectations accordingly and you might learn something.