STMFNet VFI

ST-MFNet is a quality-first interpolator in the ComfyUI Frame Interpolation pack, and like FLAVR it comes with two strings attached: it needs at least four input frames, and it only does 2x interpolation. Work within those limits and it produces some of the cleaner tweens in the pack, because it looks at more temporal context than a two-frame model can.

If interpolation is new to you: it invents new frames between your existing ones so a choppy low-fps clip plays smooth. You generate video cheaply at a low frame count with AnimateDiff or Wan and then interpolate up to a watchable frame rate, rather than paying to diffuse every real frame. STMFNet is one option for that finishing step - a picky but capable one.

How it works

ST-MFNet - "Spatio-Temporal Multi-Flow Network," CVPR 2022 - estimates multiple flow fields across a window of four frames and fuses them spatially and temporally to synthesize the middle. Using four frames instead of a single pair gives it a read on acceleration and trajectory, not just instantaneous motion, which is why its results tend to look more physically plausible on things that speed up, slow down, or curve. That four-frame window is also exactly why it can't run on two frames - there isn't enough context to feed the network.

The two constraints to remember

Straight from the pack: STMFNet needs at least 4 frames, and it can only do 2x interpolation for now (the multiplier input is locked at 2). If you only have two or three frames, the documented workaround is to interpolate up first with a different node - run FILM VFI at multiplier=4 - and then feed that longer sequence into STMFNet.

The inputs that matter

• frames (IMAGE) - your clip, at least 4 frames. This is the hard requirement.
• ckpt_name - one option, stmfnet.pth, which downloads automatically on first run.
• multiplier (INT, default 2) - fixed at 2 (min and max are both 2).
• clear_cache_after_n_frames (INT, default 10) - the OOM valve; lower it on long clips if memory gets tight.
• duplicate_first_last_frames (BOOLEAN, default false) - interpolation only fills the gaps between frames, so the endpoints don't get partnered. Turn this on to duplicate the first and last frames and keep your output count aligned instead of coming up short at the ends.

There's an optional optional_interpolation_states input for a Make Interpolation State List, to skip specific pairs. Usually left unwired.

Output is a single IMAGE batch - send it to a Video Combine node (VideoHelperSuite) to encode.

Installing it

It comes with the whole ComfyUI Frame Interpolation pack:

• ComfyUI Manager - search ComfyUI Frame Interpolation, install, restart.
• Manual - cd ComfyUI/custom_nodes && git clone https://github.com/Fannovel16/ComfyUI-Frame-Interpolation, then python install.py (Linux, in your ComfyUI venv) or install.bat (Windows). Restart.

The stmfnet.pth checkpoint downloads on first use.

Common issues

The number-one STMFNet problem is the four-frame requirement - feed it fewer and it errors out. If that's you, pre-interpolate with FILM at multiplier=4 and chain into STMFNet, exactly as the pack recommends.

The second is expecting more than 2x out of it; it won't go higher, so if you need a bigger jump, do a 2x pass here and a second interpolation pass with another node, or use a model that supports higher multipliers. Beyond that, the pack-wide stuff holds: the cupy backend is the install snag (run install.bat on Windows), a GPU sitting idle means the backend fell back to CPU, and clear_cache_after_n_frames is your lever when a long clip runs out of VRAM.