OmnimatteZero_SM_Model

OmnimatteZero is a SIGGRAPH Asia 2025 paper with a mouthful of a subtitle: "Fast Training-free Omnimatte with Pre-trained Video Diffusion Models." In plain terms, you hand it a video and a rough mask of one object, and it separates that object - plus its shadows and reflections - from the background, no fine-tuning, no new checkpoint. Then you can delete the object, or drop it onto a different scene.

smthemex's ComfyUI port runs that algorithm on top of LTX Video 0.9.7, the 13B "quality era" model from Lightricks - the fast open video model, not the drafty 2B that made the family famous. OmnimatteZero_SM_Model is the node at the start of every workflow from this pack. It's a loader, and honestly it's the boring part. It builds the LTX pipeline the whole pack runs on and hands you a MODEL.

How it works

The loader reads the diffusers configs the pack ships inside the repo (a LTX-Video-0.9.7-diffusers/ folder), then assembles the pieces of a diffusion pipeline:

• dit - the 13B diffusion transformer, picked from ComfyUI/models/diffusion_models. This is the big file.
• gguf - the same transformer as a GGUF quant from ComfyUI/models/gguf instead. The example workflow uses the Q8_0 GGUF, which is basically fp16 quality at half the RAM cost. Pick one: dit or gguf.
• vae - the LTX 0.9.7 VAE from ComfyUI/models/vae. Always required.

Notable detail: the pipeline is built with text_encoder=None. The pack ships precomputed prompt embeddings (positive.pt / negative.pt) inside the node folder, so there's no T5 text encoder, no CLIP, no prompt engineering. For a matting task that's a feature - the "prompt" is effectively baked in.

The one input that isn't model-picking is compose_mode. Leave it off for plain object removal. Flip it on when you want the compositing workflow (feeding video_bg and video_new_bg into the KSampler). That swaps in a patched VAE that encodes the original video, old background, and new background together. Manage expectations here: the README is blunt that compose mode "got a normal effect."

Install

ComfyUI Manager - search "ComfyUI_OmnimatteZero" - or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_OmnimatteZero.git

Restart ComfyUI, then install the requirements:

pip install -r requirements.txt

That requirements.txt is heavier than the README's "nothing special" shrug implies - diffusers >= 0.31, transformers, opencv, imageio-ffmpeg, and a few vestigial extras like dashscope the author admits he's too lazy to delete. The real cost is the model files:

ComfyUI/models/vae
  └── LTX-Video-0.9.7-vae-diffusers.safetensors   (from a-r-r-o-w/LTX-Video-0.9.7-diffusers)

ComfyUI/models/diffusion_models   # optional
  └── LTX-Video-0.9.7-diffusers.safetensors      (from smthem/LTX-Video-0.9.7-diffusers-merge)

ComfyUI/models/gguf              # optional, lighter
  └── LTX-Video-0.9.7-diffusers-Q8_0.gguf

Output

One output: model (MODEL). Wire it straight into OmnimatteZero_SM_KSampler - that's the node that does the actual work.

Gotchas

• This is a big-model node. The author tested it on 12GB VRAM / 64GB RAM at 1280x720 5s; on smaller cards grab the GGUF and keep block_num low in the KSampler.
• The pack imports specific diffusers pipeline internals, so a ComfyUI or diffusers update can break it. If it stops loading, update the pack.
• People do hit "missing node" errors right after install - that's the requirements step in practice, so install them before you panic.