TLBVFI Frame Interpolation

Most frame interpolation in ComfyUI is optical-flow guesswork. RIFE, GIMM, FILM - they look at two frames, estimate where pixels moved, and warp something into the gap. Fast, and usually great. TLBVFI is not that. It runs a full latent diffusion model - a Brownian Bridge sampler in VQGAN latent space - to generate the in-between frame from scratch. It's the "motion interpolation, but as a diffusion model" paper (TLB-VFI, arXiv 2507.04984) wrapped for ComfyUI, and it behaves exactly like a diffusion model: slower than the flow-based crowd, pickier about hardware, but capable of motion those methods mangle.

Why would you reach for it? Two cases. You have genuinely fast, complex motion - a whip-pan, water, cloth - and RIFE's warped smearing bugs you. Or you're pushing slow-motion hard: times_to_interpolate at 3–4 means you're inventing several frames between every real pair, and each invented frame is where flow-based methods degrade. Diffusion has no "best guess" bias the same way; it has context. One honest caveat from the community: that context is a double-edged sword. Where the model doesn't understand the motion it produces artifacts - someone put it bluntly: RIFE is unbiased and TLB is biased, so pick your poison per clip.

How it works

Three stages, straight from the source:

1. VQGAN encoder compresses your two frames into a latent space.
2. A Brownian Bridge UNet diffuses between the two latents - instead of just interpolating, it learns the stochastic path from frame A to frame B and samples a midpoint.
3. VQGAN decoder pops the result back to full-res frames.

That's why diffusion_steps exists. Every step is a refinement pass, like a sampler step count. The README's defaults are sane: 10 steps is decent, 20–50 if you want quality, and it's a direct speed trade.

The inputs that matter

The schema is small - that's the charm. Feed it frames and a model:

• images - a batch of frames as IMAGE, i.e. at least two. Feed it from VHS Load Video/Load Frames output. Fewer than two frames and it just hands your input back.
• model_name - dropdown, populated by scanning ComfyUI/models/interpolation/ for .pth files.
• times_to_interpolate (default 1, max 4) - how many frames to invent per pair. 1 doubles FPS, 2 quadruples it.
• diffusion_steps (default 10, max 100) - the sampler refinement. This is your quality knob.
• batch_size (default 2) - frame pairs processed at once. Raise it on a beefy card.
• flow_scale (default 0.5) - resolution of the motion analysis. Drop to ~0.3 for fast motion, raise toward 1.0 if fine detail is shifting.

Output is a single IMAGE batch - the original frames plus the invented ones in order - which you feed to a VHS Video Combine to write the file.

Installing it

ComfyUI Manager finds it by searching ComfyUI-TLBVFI (it registers under "TLBVFI Frame Interpolation"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/BobRandomNumber/ComfyUI-TLBVFI.git

Restart, then the one thing everyone trips on: the model is not bundled. Grab vimeo_unet.pth from ucfzl/TLBVFI on Hugging Face and drop it at:

ComfyUI/models/interpolation/vimeo_unet.pth

No model, no luck - the node errors out with "No TLBVFI UNet models found." Good news on dependencies: the pack is genuinely zero-dependency. The original project needed CuPy and PyTorch-Lightning; this wrapper ripped them out for native implementations, so pyproject.toml has an empty dependency list. If you have ComfyUI, you have what this needs.

Gotchas

• It's slow. Diffusion sampling per pair of frames, no way around it. This is a render-and-go-make-coffee node, not a realtime one. Don't set diffusion_steps at 50 and wonder why your card is screaming.
• Watch VRAM. Each pair runs a full VQGAN + UNet pass; times_to_interpolate multiplies the work, and the sampled frames are cached in memory. High res + high steps on a low-VRAM card will OOM.
• Artifacts where context fails. Occlusion-heavy or chaotic motion is where the "biased" diffusion guess shows. If a clip comes out warped, drop flow_scale a notch - the lower motion-analysis resolution genuinely helps fast motion.

Is this the one to reach for daily? No. RIFE is faster and often just as good, and that's what most workflows should use. But TLBVFI is the heavy artillery for the shots RIFE can't win - and for anyone curious how diffusion does motion, it's a surprisingly clean look at the mechanism.