OmniLottie_SM_Model

In ComfyUI, a MODEL output usually means a diffusion transformer you're about to run through an actual KSampler. OmniLottie_SM_Model breaks that expectation on purpose. It loads the OmniLottie decoder: a Qwen2.5-VL-3B vision-language model that was fine-tuned to write Lottie animation code - the JSON vector format the web and After Effects both eat - directly as output tokens. The node itself is just a loader, but it's the piece that decides whether the rest of this pack does anything.

What you're actually loading

The pack (smthemex's port of OpenVGLab's OmniLottie, arXiv 2603.02138) doesn't ship the weights in the repo. It ships a Qwen2.5-VL-3B-Instruct folder with the config and tokenizer files, and it builds the model architecture from those, then loads the actual weights from a merged checkpoint you drop into ComfyUI/models/diffusion_models/. That merge is the whole trick: the author took the official OmniLottie weights and merged them with the Qwen2.5-VL base so you load one .safetensors instead of assembling pieces. Grab Lottie_weight.safetensors from smthem/OmniLottie-merge on Hugging Face - the README points there (and at a cloud drive if you can't reach HF).

It's a 3B VLM, not a 20B diffusion transformer, so VRAM is genuinely modest - a few GB, well within reach on the same card that runs SDXL. What it's not: fast. "Sampling" here is autoregressive token generation, LLM-style, so your ComfyUI queue sits for a bit while it writes out JSON tokens. That's not a hang, it's how this model works.

The inputs that matter

There are only three, and two of them you can ignore:

• ckpt - a dropdown of everything in your diffusion_models folder. Pick Lottie_weight.safetensors. The list opens with "none" as a sentinel, but there's no real no-model path - the code assumes a file path and will error on "none". Select the actual checkpoint.
• pix_len (default 1500) - the token budget for the Lottie JSON the model is allowed to write. This is the one knob worth knowing: if your animations come back chopped off mid-scene, raise it. It's literally max_new_tokens at generation time, so bigger means slower.
• text_len (default 4096) - the max prompt/context length in tokens. Leave it alone.

Install

Two steps beyond the normal clone, and both matter. First, get the pack in - via ComfyUI Manager (search "ComfyUI_OmniLottie") or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_OmniLottie.git
pip install -r requirements.txt

Then restart ComfyUI and put the model where the node looks for it:

ComfyUI/models/diffusion_models/Lottie_weight.safetensors

That path is the trap. The dropdown is built from diffusion_models specifically, not checkpoints - if you plop the file in the wrong folder, the ckpt list won't even show it. And the dependencies (decord, opencv-python, qwen-vl-utils, datasets) are heavier than the average custom node's, so give pip a moment and expect a few conflicts if you already have a packed environment. That's the standard ComfyUI dependency pain, nothing unique here.

What comes out

A single model output, wired straight into OmniLottie_SM_KSampler. There's nothing else to connect - no CLIP, no VAE, no empty latent. That's the tell: you're not doing diffusion math, you're handing a text-to-vector generator its pen. If the node is missing from your menu after install, check that diffusion_models contains the checkpoint - the loader is the boring but mandatory first step of this whole pack.