Nodes/ComfyUI_LTX2_SM/LTX2_SM_Clip
ComfyUI Node

LTX2_SM_Clip

Loading LTX-2.3's Gemma 3 text encoder without it eating your GPU

By smthemex·Created 7 months ago·Updated 3 months ago· 31
LTX2_SM_Clip
    • clip
    clip
    connector
    infer_device

    The name is doing a lot of work here. LTX2_SM_Clip isn't loading a CLIP model - LTX-2.3 doesn't use CLIP at all. It's loading the Gemma 3 12B text encoder as a quantized GGUF plus the connector.safetensors bridge, and presenting both as a single CLIP-typed object so the rest of the graph can treat it like a text encoder. It's the node that decides whether your memory budget survives.

    Why Gemma is the fight

    LTX-2.3 pairs its 22B DiT with a 12B Gemma 3 text encoder, and in full fp16 that thing is ~22GB - historically the number-one cause of OOM errors on LTX-2, worse than the video model itself. The whole point of this pack is running it small: the README's file is gemma-3-12b-it-qat-Q4_0.gguf, a ~7GB quant that lives in ComfyUI/models/gguf/. Your clip dropdown lists GGUF files, not checkpoints.

    The three inputs:

    • clip - the Gemma GGUF (from models/gguf/).
    • connector - connector.safetensors, from models/checkpoints/. This is the "text connector" - not a text encoder but the bridge between Gemma's output and the generation model's conditioning. LTX-2.3 made it bigger than 2.x's, which is part of why multi-subject prompts behave better.
    • infer_device - cuda or cpu. Here's the move: on anything under ~16GB VRAM, run Gemma on CPU. It's slow to load and slow to encode, but it keeps the GPU free for the DiT, and with LTX2_SM_ENCODER's save_emb you only pay that cost once per prompt.

    The output clip feeds straight into LTX2_SM_ENCODER. There's no way around this node - Gemma is mandatory for LTX-2.3 - but there's also no reason to let it own your card.

    Gotchas

    • The tokenizer is bundled. The pack ships the Gemma tokenizer config inside LTX2/gemma/, so you don't download a separate tokenizer. If loading fails with a tokenizer error, that folder got deleted or the clone is incomplete - re-clone.
    • Run it on CPU if you're on 6–12GB. You'll wait longer per encode; you'll OOM less. With save_emb on in the encoder, it's a one-time wait per prompt.
    • Q4 is the community norm, not a compromise. Unsloth's 4-bit Gemma is what people actually run for 2.3; the 22GB fp16 version is the trap.

    Installing

    Standard for this pack - ComfyUI Manager (search "ComfyUI_LTX2_SM") or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/smthemex/ComfyUI_LTX2_SM.git
    cd ComfyUI_LTX2_SM && pip install -r requirements.txt
    

    Requirements are diffusers, accelerate, transformers, tokenizers, gguf, and omegaconf. Download the Gemma GGUF into ComfyUI/models/gguf/ and the connector into ComfyUI/models/checkpoints/, restart, and both dropdowns populate. Missing from the list means the file isn't where ComfyUI is looking.

    CategoryLTX2_SM

    Inputs (3)

    NameTypeDefaultDescription
    clipCOMBO1 options: none
    connectorCOMBO1 options: none
    infer_deviceCOMBO2 options: cuda, cpu

    Outputs (1)

    NameTypeDescription
    clipCLIP