Spline Runner

Back before video diffusion was a thing, this is the model people actually used to animate generated faces. The classic r/StableDiffusion pipeline of late 2022 - the +1500-point "animating generated face" post - was literally Thin-Plate-Spline → upscale → GFPGAN face-fix. If you recognize the idea of driving a portrait with a video's motion, Spline Runner is the direct upgrade to the FOMM Runner: same job, same 256×256 output, but built on thin-plate-spline (TPS) transformations instead of raw keypoint warping. The result is smoother deformation, less of that wobbly GAN stretch, and generally cleaner motion - which is why the TPS model (yoyo-nb's Thin-Plate Spline Motion Model, 2022) quickly became the community favorite over plain FOMM.

In this pack, Spline_Runner is the one you reach for when FOMM looks too rubbery and you don't need a transformer's worth of compute. It's still seconds-per-video-fast, still a small checkpoint, still no sampling step.

How it works

Very much the FOMM playbook, refined. A keypoint detector finds sparse keypoints on the source and each driving frame, a dense-motion network builds a dense flow field from them, and an inpainting network renders the warped result. The difference is the motion representation: instead of just translating keypoints, TPS fits smooth splines through them, which is what kills the jittery, piecewise artifacts FOMM is known for.

The inputs that matter

There are only two knobs, plus the standard source_image, driving_video_input (a batch of frames - feed it from VHS_LoadVideo), frame_rate, and pass-through audio:

• predict_mode - the one that matters.
  • relative (default) - inherits the source's geometry and scales motion to it. Closest to FOMM with relative_movement + adapt_movement_scale on.
  • standard - the source gets mapped straight onto the driver's keypoints. Closest to FOMM with adapt_movement_scale off; expect more distortion if the faces don't match.
  • avd - runs the driving motion through an additional learned AVD network on top of relative mode. Can look better, but the README warns it's more "jittery/jumpy." Try it; it's one dropdown.
• find_best_frame - same feature as FOMM, same catch: it needs the optional pip install face-alignment or it silently does nothing and starts at frame 0.

Outputs are the pack standard: images (the frame batch), audio (untouched pass-through), and frame_rate. Wire all three into VHS_VideoCombine.

Install and the one big gotcha

Installation is the shared pack story - ComfyUI Manager, search "FirstOrderMM", or:

cd ComfyUI/custom_nodes
git clone https://github.com/FuouM/ComfyUI-FirstOrderMM.git
pip install -r requirements.txt

The dependencies are light (numpy, torch, scipy, pyyaml, matplotlib, einops, timm - no OpenCV, no PyAV). Restart and the nodes are under FirstOrderMM.

The model is not auto-downloaded. Spline needs module_spline/models/vox.pth.tar, fetched from the Thin-Plate-Spline-Motion-Model's pre-trained models page, dropped in that exact folder. Miss it and the node dies on load with a file-not-found. There's no model_name dropdown here - there's only the one vox checkpoint, so the filename is fixed. Get it right once and forget it.

Common issues

• Rubbery or stretching faces in standard mode - switch to relative; it scales motion to the source instead of forcing the source into the driver's geometry.
• Jittery results in avd mode - that's expected behavior, not a bug. Drop back to relative if it bothers you.
• 256×256 output looks soft - yes. Upscale after, or use this as the motion stage of a bigger pipeline (which is exactly what people did back in 2022).
• find_best_frame no-ops - pip install face-alignment and let it fetch its models.