Pertineo ViTMatte Tiled Refiner

ViTMatte is one of the good ones. Give it a plate and a decent trimap and it pulls clean edges out of hair, motion blur and semi-transparency in a way segmentation models structurally can't - they hand you a probability of "foreground", it hands you fractional alpha.

Then you try it at 4K and everything falls apart. A single full-res pass doesn't fit in VRAM, so most implementations quietly downscale, run at 1024, and upscale the result - which throws away precisely the edge detail you bought the model for. So people tile it by hand, and then the seams show up, because each tile is matted with slightly different context and the two answers disagree along the border.

Pertineo ViTMatte Tiled Refiner is the version that solves both. Overlapping tiles, cosine-weighted blending, and it never downscales.

The mechanism, because it matters

The plate is cut into tile_size squares stepping across the image with overlap pixels of shared edge, and each tile is matted independently. Then, instead of picking a winner per border, every tile contributes to an accumulator with a 2D weight - flat 1 through the middle, ramping down across the overlap as a half-cosine, deliberately never reaching zero. Weighted sum, divide by summed weights: each pixel is a smooth average of every tile that saw it. There's no hard line because there's no line.

The trick that makes it fast on real footage: tiles with no unknown area are skipped entirely and just pass the trimap through. If your subject covers a third of the frame, most tiles do no inference. The console reports tiles computed vs. total - the fastest way to diagnose a broken trimap.

flip_tta runs each tile a second time mirrored and averages the two. A real gain on fine edges - calmer, less wobbly - for exactly twice the time.

The inputs

Two that carry the work:

• image - the full-resolution plate batch, RGB, sRGB.
• trimap - 0 background, 0.5 unknown, 1 foreground. Resized to the plate if the sizes differ, so a mismatched pair doesn't crash, it just gets resampled.

Four that you'll actually tune:

• model - base (best quality) or small (faster); custom loads your own weights.
• tile_size - the biggest number that fits your card. More context per tile = better matte. VRAM grows quadratically: roughly 1536 at 32 GB, 2048 does not fit.
• overlap - how much tiles share. More overlap, smoother transitions, slightly slower. If you can still see a grid, raise this before anything else.
• flip_tta / clamp_known / quantize_trimap - leave on to start. quantize_trimap snaps the trimap to 0/0.5/1 (cleans dithered or colour-shifted trimaps); turn it off only if you're deliberately feeding a soft trimap. clamp_known forces 0/1 outside the unknown band, which removes noise but also means trimap errors can't be rescued.

custom_model is optional and only read when model = custom: an absolute path, or a subfolder name inside ComfyUI's models/vitmatte folder, containing config.json + model.safetensors.

One output: alpha, a MASK batch. Wire it into Pertineo Save Matte Sequence, or straight into a mask composite.

Install

ComfyUI Manager → Pertineo VFX, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/pertineo/comfyui-pertineo-vfx
pip install -r comfyui-pertineo-vfx/requirements.txt

OpenEXR>=3.2 is the pack's only dependency. Weights are separate and not in the repo: the first time you run it, the node pulls the official hustvl/vitmatte-base-composition-1k (or -small-) weights into ComfyUI/models/vitmatte using huggingface_hub, so there's no manual download. If you use extra_model_paths.yaml, a vitmatte: entry is picked up too.

Where people get burned

Out of VRAM. Drop tile_size in steps of 256 and restart ComfyUI - the model is cached per process, and lowering the tile size without a restart often doesn't free what you need.

The licence, which is the real trap. The node is GPL. The default weights are Apache-2.0 on Hugging Face but were trained on Composition-1k, which is built from the Adobe Deep Image Matting dataset - and that restricts models trained on it to non-commercial use. Other projects using these same weights say the same thing. If you're doing paid client work with the default weights, sort that out with your client first. The node prints a licence reminder to the console every time it loads the hustvl weights, which is not noise, it's a warning. Licensed or self-trained weights go in via custom.

RAM, not VRAM. A 4K IMAGE batch runs about 100 MB per frame. A long shot will exhaust system RAM long before the GPU complains. Use Pertineo ViTMatte Tiled (Sequence) for anything feature-length.

Expecting a tutorial ecosystem. ViTMatte barely exists in community chatter - a handful of threads a year, mostly from compositors. That's not a red flag about the model, just about expecting a YouTube workflow to cover it.