SGM-VFI Interpolate

Most frame interpolation assumes things move a little between frames. SGM-VFI Interpolate is the node you pull out when things move a lot - a whip-pan, a fast ball crossing the frame, sports footage, a drone flipping over. Pairwise models that rely on local flow estimation smear or ghost on those; SGM-VFI (CVPR 2024) does sparse global matching to find where regions of the frame went, so large displacements stay coherent instead of turning into mush. It's the pack's designated "hard motion" tool.

The trade-off is the honest one: it's the slowest pairwise option in the pack and heaviest at ~3GB per frame pair. Don't use it as your everyday smoother - that's what EMA-VFI (speed) or BIM-VFI (balanced) are for. Reach for it when your footage has the kind of motion that makes the others look bad, and you'll understand why it exists.

Inputs. Same shared pairwise control set as the rest of the pack:

• images + model (from Load SGM-VFI Model) - the essentials.
• multiplier - 2x/4x/8x via recursive passes (2N−1, 4N−3, 8N−7 outputs).
• batch_size - pairs per pass; start 1, raise until VRAM fills. chunk_size - segment the input to bound VRAM. keep_device / all_on_gpu - the usual speed-for-VRAM trades; all_on_gpu wants a big card.
• source_fps / target_fps - set both for exact-FPS output (e.g. 24→60): the node oversamples to a power-of-2 and picks the nearest frame per target timestamp, beating multiplier.
• clear_cache_after_n_frames - worth paying attention to here, since each pair is expensive; a lower value keeps long runs from creeping into OOM.

The model's motion behavior itself is set at the loader, not here - num_key_points on Load SGM-VFI Model controls global-matching sparsity (lower = more global matching, better on extreme motion, slower). You tune that once per footage type.

Outputs: images (finished batch → VHS Video Combine), oversampled (pre-FPS-selection frames), and elapsed_seconds.

Install. Search "Tween" in ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Ethanfel/ComfyUI-Tween
pip install -r requirements.txt

Restart, let the loader grab the checkpoint, and consider pip install cupy-cuda12x (match your CUDA) - SGM benefits most from cupy's accelerated warping, though it has a PyTorch fallback.

Where people get burned: mostly on expectations - it's slow, so don't benchmark it against EMA-VFI and assume something's broken. If hard-motion footage still smears, drop num_key_points toward 0 at the loader before giving up. And for very long clips, this is the node where you want the Segment variant or at least a sane chunk_size, because slow + huge is how you spend an afternoon.