MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
-
Clone this repo into custom_nodes directory of ComfyUI location
-
Run pip install -r requirements.txt
-
Download the weights of MotionCtrl motionctrl.pth and put it to
ComfyUI/models/checkpoints
Four nodes Motionctrl Sample
& Load Motion Camera Preset
& Load Motion Traj Preset
& Select Image Indices
Motion Traj Tool Generate motion trajectories
base workflow
https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_threenodes.json
unofficial implementation "MotionCtrl deployed on AnimateDiff" workflow:
https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl.json
- Generate LVDM/VideoCrafter Video
- Select Images->Scribble
- Use AnimateDiff Scribble SparseCtrl