OmniLottie_SM_KSampler

The name is a lie. OmniLottie_SM_KSampler has "KSampler" in the title but there is no diffusion sampling happening anywhere in it - no latent, no denoising steps, no CFG scale to fiddle with. What it actually does is autoregressive token generation: a Qwen2.5-VL-based decoder writes a Lottie JSON animation, token by token, from your prompt (or from an image or video you feed it), and hands you back the file path plus a rendered HTML preview. Lottie is the vector-animation JSON format that runs natively in web and mobile apps and opens in After Effects - so instead of a pixel video, you get genuinely editable, resolution-independent vector motion.

Why you'd reach for it

A bouncing coin, a loading spinner, a character's idle loop, a logo reveal you can ship to a frontend without a video encode step. That's the niche: text-to-vector-animation, no raster frames involved. It's a research-model port (OmniLottie from OpenVGLab), so set expectations accordingly - it's impressive when it lands, and it's new enough that it's a toy more than a production tool.

Inputs that matter

• model - from OmniLottie_SM_Model. There's no other wiring; no CLIP, no latent.
• prompt (multiline) - describe the animation as a timed scene. The default is the paper's piggy-bank demo, and yes, the currency symbol in it is mangled mojibake - you'll overwrite it with your own text anyway. In text mode this prompt is the whole job: name the objects, their motion, and the order it happens.
• temperature (0.9), top_p (0.25), top_k (5) - LLM-style sampling knobs, not diffusion ones. The defaults are already tuned for coherent Lottie output; if generations come out repetitive or locked, nudging temperature up is the dial you want.
• images (optional) - plug in a single image for image-to-Lottie, or a batch of frames for video-to-Lottie. The node checks the batch dimension: one frame means image mode (upscaled to 448px), more than one means video mode (frames to 336px). So a LoadVideo → GetVideoComponents chain from VideoHelperSuite gives you video mode, and LoadImage gives you image mode. The README's own advice: in image mode, still add a descriptive prompt - image-only results are weak.

Outputs

• path - the filesystem path to the generated Lottie JSON in ComfyUI's output folder (single_text_result_<timestamp>.json, and similarly for image/video). This is the actual deliverable.
• html - a path to a self-contained HTML page that renders the animation in a browser. ComfyUI has no native Lottie preview, so this is the fastest way to see what you generated. Wire both into PreviewAny and open the files.

Install

Same pack steps as the model node - Manager (search "ComfyUI_OmniLottie") or:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_OmniLottie.git
pip install -r requirements.txt

Restart, then drop Lottie_weight.safetensors (from smthem/OmniLottie-merge on Hugging Face) into ComfyUI/models/diffusion_models/. The requirements.txt pulls decord, opencv-python, qwen-vl-utils, and datasets - heavier than most custom nodes, so expect pip to churn a bit.

Common issues

• "Nothing previews." The output is a JSON file and an HTML file, not an image - open the html path in a browser, or add PreviewAny.
• Weak image-to-Lottie results. Add a descriptive prompt alongside the image; the README calls this out explicitly.
• Video mode not kicking in. You need more than one frame in the images input. A single LoadImage output is batch 1, which is image mode - feed a frame sequence for animation.
• Animations cut off mid-scene. Raise pix_len on the model node; the sampler's generation token budget is what truncates.
• It just takes a while. A 3B VLM writing up to ~1500 tokens of JSON isn't instant. A minute in the queue is normal, not a crash - give it room before you start hunting for errors.