ComfyUI-MotionCtrl
Nodes: Download the weights of MotionCtrl a/motionctrl.pth and put it to ComfyUI/models/checkpoints
Nodes (7)
27 ready-made camera moves you can paste straight into your prompt
Where MotionCtrl's weights actually load
Curves, shakes and horizons without opening the drawing tool
Prompt, camera pose and trajectory in, conditioning out
Everything in one node, and it shows
The sampler that actually makes the video — once the graph is wired
The boring node that makes MotionCtrl + AnimateDiff scribble work
This is an implementation of MotionCtrl for ComfyUI
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Install
-
Clone this repo into custom_nodes directory of ComfyUI location
-
Run pip install -r requirements.txt
-
Download the weights of MotionCtrl motionctrl.pth and put it to
ComfyUI/models/checkpoints
Nodes
Four nodes Load Motionctrl Checkpoint & Motionctrl Cond & Motionctrl Sample Simple & Load Motion Camera Preset & Load Motion Traj Preset & Select Image Indices &Motionctrl Sample
Tools
Motion Traj Tool Generate motion trajectories
<img src="assets/traj.png" raw=true>Motion Camera Tool Generate motion camera points
<img src="assets/camera.png" raw=true>Examples
base workflow
<img src="assets/base_wf.png" raw=true>https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl_base.json
<video controls autoplay="true"> <source src="assets/dog.mp4" type="video/mp4" /> </video>unofficial implementation "MotionCtrl deployed on AnimateDiff" workflow:
<img src="assets/scribble_wf.png" raw=true>https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl.json
- Generate LVDM/VideoCrafter Video
- Select Images->Scribble
- Use AnimateDiff Scribble SparseCtrl