LTX2AdvancedCondition

LTX-2's biggest weakness at launch wasn't the model - it was that the prompt box is blind. The 19B DiT runs on a Gemma 3 12B text encoder, and while that encoder can genuinely read an image, ComfyUI's stock LTX-2 workflow never hands it one. This node is someone's attempt to fix that: it lets you drop a literal <start_of_image> tag into your prompt and have the model see the picture at that exact spot, mid-sentence. Think of it as attaching an image to a chat turn, except the chat is your video prompt and the reply is a conditioning tensor.

It's an experimental hack, and the author says so in the README: it does something, but the conditioning picks up artifacts and "it even affects the audio." You're reading this because you're curious or stubborn - good. Just know what you're signing up for.

How it works

It's a drop-in replacement for the text-encode step in an LTX-2 workflow. Feed it the Gemma 3 CLIP, a prompt, and an image, and it:

1. Tokenizes your prompt with the Gemma 3 tokenizer.
2. Finds the <start_of_image> token (id 255999) in the gemma3_12b token stream.
3. Runs the image's embeddings through Gemma3MultiModalProjector to produce however many tokens you asked for.
4. Replaces the tag with those embedding vectors, right there in the sequence.
5. Encodes the whole mixed stream into CONDITIONING.

So the text tokens around the tag get to attend to image-derived tokens - mechanically identical to how Gemma 3 sees an image when one is attached to a chat prompt. That hard-coded gemma3_12b stream is also why it only works with the Gemma 3 12B encoder; feeding it an SD/SDXL CLIP or a T5 will not end well.

The inputs that matter

• prompt - where the magic happens. Put <start_of_image> wherever you want the image "inserted." The default is the author's dry little demo: Cinematic scene where <start_of_image> stands in front of the camera saying 'Hello World!' happily. Describe what the picture shows around the tag and the model fills the gap.
• image (CLIP_VISION_OUTPUT) - the output of the stock CLIP Vision Encode node, fed by Gemma3VisionLoader's CLIP_VISION plus your image. It's marked optional, but a <start_of_image> tag with nothing wired in has nothing to inject - leave the tag out if you're not feeding an image.
• tokens_per_image - default 5, range 1–256. How many embedding tokens the projector emits per image. The author's tooltip says it plainly: "Number of tokens to produce to describe each image." More tokens = a richer picture in the stream, but also more signal to go wrong; start at the default.

Output: a single CONDITIONING, which you wire into the LTX-2 sampler in place of the usual text encode output.

Install

Same story as its sibling node: no dependencies beyond a current ComfyUI, and you need the Gemma 3 12B safetensors in models/text_encoders/ (the same 22GB-ish file you already have for LTX-2).

cd ComfyUI/custom_nodes
git clone https://github.com/nistvan86/comfyui-inagy-ltxtools

Restart, or grab it via ComfyUI Manager under "comfyui-inagy-ltxtools".

Common issues

• NaN errors. The signature bug of this pack: Gemma3MultiModalProjector sometimes produces an all-NaN tensor. The node actually raises a helpful ValueError telling you to clear the model cache - do that (or restart ComfyUI) and it goes away until it doesn't. The author suspects ComfyUI's model caching.
• "Conditioning tensor got lost somewhere" - same family, same fix.
• Artifacts and audio weirdness. Not a bug, it's the feature. The conditioning signal is dirty and it bleeds into LTX-2's audio generation. If you're chasing clean results this isn't your node.

Where this leaves you: if you just want image-conditioned LTX-2, Lightricks' own tooling (and the community workflows around it) will serve you better. But if you want to actually understand how Gemma 3's multimodal conditioning slots into a video model, this is a great sandbox - the code is short enough to read in one sitting, and the failure modes are documented honestly.