Nodes/ComfyUI-FirstOrderMM/Articulate Runner
ComfyUI Node

Articulate Runner

Source, video, done

By FuouM·Created 2 years ago·Updated about a year ago· 9
Articulate Runner
  • source_image
  • driving_video_input
  • audio
  • images
  • audio
  • frame_rate
frame_rate30.00

Articulate Runner is the easiest node in this pack, and it's easy on purpose. It runs Articulated Animation (snap-research, Siarohin et al., CVPR 2021) - the successor paper to First Order Motion Model - and where FOMM_Runner gives you three movement toggles to fiddle with, this one gives you nothing. The README literally lists "Doesn't need any" under its arguments. You wire in a source image, wire in a driving video, and it animates.

The draw over plain FOMM is the motion representation. FOMM models faces as ten sparse keypoints, which is why it goes rubbery on big head turns. Articulated Animation replaces those with motion regions - small learned patches that each carry an affine transformation - so the model can represent fine-grained, local deformation much more naturally. The result is better-looking motion on larger pose changes, with the same GAN speed and the same 256×256 output. If FOMM looks stiff and FSRT is overkill, this is the middle ground.

How it works

A region predictor estimates the motion regions (with their affine transforms) on the source and on each driving frame. The code then computes the animation in "relative" mode - it measures how each driving region moved relative to its starting frame, scales that movement to the source's geometry via a convex-hull area ratio, and applies it to the source regions - and an AVD network (motion transfer module) refines the result before the generator renders each frame. All of it hardcoded: no knobs, no mode switch, no find_best_frame. That's why the node's input list is just source_image, driving_video_input, frame_rate, and the optional pass-through audio.

The inputs (all of them)

  • source_image - a single still (a batch works, but you're animating one face per run).
  • driving_video_input - a batch of frames. Feed it from VHS_LoadVideo, which is what the pack's own ARTICULATE workflow does.
  • frame_rate - default 30. Mostly cosmetic; it just rides through to the output.
  • audio (optional) - untouched pass-through, so you can route the driving video's audio into VHS_VideoCombine alongside images.

Outputs are images, audio, frame_rate, wired into VHS_VideoCombine exactly like the other runners.

Install

The pack is one install for everything - ComfyUI Manager (search "FirstOrderMM") or:

cd ComfyUI/custom_nodes
git clone https://github.com/FuouM/ComfyUI-FirstOrderMM.git
pip install -r requirements.txt

Restart and it's under FirstOrderMM. Dependencies are light (numpy, torch, scipy, pyyaml, matplotlib, einops, timm; no OpenCV, no PyAV).

The one thing you must do manually: put the checkpoint at module_articulate/models/vox256.pth, from the Articulated Animation pre-trained checkpoints page. There's no dropdown and no fallback - the node hardcodes that path. No model, no output, and the failure is a plain file-not-found.

Common issues

  • Missing file on load - module_articulate/models/vox256.pth isn't placed. That's the whole failure mode for this node.
  • Motion looks weak - that's relative mode doing its thing: it inherits the source's proportions rather than forcing the driver's exact pose. It's the intended tradeoff for stability. You can't change it here; this node is deliberately locked down.
  • 256×256 output - expected. Treat it as the motion stage, then upscale.
CategoryFirstOrderMM

Inputs (4)

NameTypeDefaultDescription
source_imageIMAGE
driving_video_inputIMAGE
frame_rateFLOAT30.00
audiooptAUDIO

Outputs (3)

NameTypeDescription
imagesIMAGE
audioAUDIO
frame_rateFLOAT