demo.mp4
Official implementation of FreeDrag: Point Tracking is Not You Need for Interactive Point-based Image Editing.
- Authors: Pengyang Ling*, Lin Chen*, Pan Zhang, Huaian Chen, Yi Jin
- Institutes: University of Science and Technology of China; Shanghai AI Laboratory
- [Paper] [Project Page] [Web Demo]
This repo proposes FreeDrag, a novel interactive point-based image editing framework free of the laborious and unstable point tracking process🔥🔥🔥.
To serve the intricate and varied demands of image editing, precise and flexible manipulation of image content is indispensable. Recently, DragGAN has achieved impressive editing results through point-based manipulation. However, we have observed that DragGAN struggles with miss tracking, where DragGAN encounters difficulty in effectively tracking the desired handle points, and ambiguous tracking, where the tracked points are situated within other regions that bear resemblance to the handle points. To deal with the above issues, we propose FreeDrag, which adopts a feature-oriented approach to free the burden on point tracking within the point-oriented methodology of DragGAN. The FreeDrag incorporates adaptive template features, line search, and fuzzy localization techniques to perform stable and efficient point-based image editing. Extensive experiments demonstrate that our method is superior to the DragGAN and enables stable point-based editing in challenging scenarios with similar structures, fine details, or under multi-point targets.
[2023/7/15] Code of local demo is available now!💥
[2023/7/11] The paper and project page are released!
- Local demo of FreeDrag
- Web demo of FreeDrag
- Diffusion-based FreeDrag
- FreeDrag anything 3D
First clone our repository
git clone --depth=1 https://github.com/LPengYang/FreeDrag
Then download the pre-trained models of stylegan2
bash download_models.sh
Finally initialize the gradio platform for interactive point-based manipulation
CUDA_LAUNCH_BLOCKING=1 python FreeDrag_gradio.py
If you find our work helpful for your research, please consider citing the following BibTeX entry.
@article{ling2023freedrag,
title={FreeDrag: Point Tracking is Not You Need for Interactive Point-based Image Editing},
author={Ling, Pengyang and Chen, Lin and Zhang, Pan and Chen, Huaian and Jin, Yi},
journal={arXiv preprint arXiv:2307.04684},
year={2023}
}