MRFA Runner

MRFA Runner runs Learning Motion Refinement for Unsupervised Face Animation (Tao et al., NeurIPS 2023), and if you've used plain FOMM you'll recognize the pitch: same keypoint-based image animation, but with one specific complaint fixed. FOMM's warping tends to blur the face and drift the identity over a clip. MRFA adds a motion-refinement stage - built on RAFT optical flow - that cleans up the motion field before generation, so you get sharper features and less identity drift. It's still a 256×256 GAN and still fast, but it's the version you pick when you want FOMM's speed without FOMM's mush.

It sits between the extremes in this pack: more refined than FOMM and Articulate, much cheaper than the FSRT transformer. Think of it as "FOMM, but the 2023 retake."

How it works

The architecture is FOMM's extended: an encoder/keypoint detector produces keypoints, a dense-motion network builds the dense flow, and a decoder renders the frame - but the dense motion gets passed through a refinement module that uses optical flow (the raft.py in module_mrfa/) to make the warp consistent and coherent instead of letting each keypoint do its own thing. The two model choices reflect the training data: vox (default, the VoxCeleb talking-head dataset) and celebvhq (CelebA-HQ, which per the README is better for higher-quality faces).

The inputs that matter

• model_name - vox or celebvhq. Start with vox; switch to celebvhq for cleaner celebrity faces.
• use_relative - the big one. On (default), the model uses relative keypoint mode - the FOMM-style displacement-and-scale approach that inherits the source's proportions. Off, it runs in absolute mode, which is closer to FOMM with adapt_movement_scale set to false: the source gets mapped directly onto the driver's geometry, more faithful to the driver but rougher on mismatched faces.
• relative_movement, relative_jacobian, adapt_movement_scale - these only take effect when use_relative is on, and they're the same trio as FOMM: relative keypoint displacement, Jacobian carry-over (leave on), and convex-hull area scaling (default off here, unlike FOMM - the author tuned it that way for a reason, so don't feel compelled to flip it).

The rest is pack standard: source_image, driving_video_input (a batch of frames, feed it from VHS_LoadVideo), frame_rate, and optional audio that passes through untouched. Outputs are images, audio, frame_rate, wired into VHS_VideoCombine.

Install

One pack, one install - ComfyUI Manager (search "FirstOrderMM") or:

cd ComfyUI/custom_nodes
git clone https://github.com/FuouM/ComfyUI-FirstOrderMM.git
pip install -r requirements.txt

Light deps (numpy, torch, scipy, pyyaml, matplotlib, einops, timm), restart, nodes under FirstOrderMM.

Manual model download, as with everything here: module_mrfa/models/vox.pth for the default, celebvhq.pth if you want the other option. Both come from the MRFA pre-trained checkpoints page. The dropdown only lists names - the pack never fetches the weights for you.

Common issues

• Missing file - module_mrfa/models/vox.pth (or celebvhq.pth) isn't in place.
• Face mapped onto the driver's geometry too aggressively - use_relative is off, or adapt_movement_scale got enabled. Relative mode is the stable default.
• Identity drift on long clips - it's much better than FOMM here, but 256×256 is still 256×256. Keep clips short and consider an upscale pass.