/SemFlow

Primary LanguagePythonMIT LicenseMIT

SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow

Chaoyang Wang1  Xiangtai Li2  Lu Qi3  Henghui Ding2  Yunhai Tong1  Ming-Hsuan Yang3
1PKU, 2NTU, 3UC Merced

• [arXiv] •

Introduction

We present SemFlow, a unified framework that binds semantic segmentation and image synthesis via rectified flow. Samples belonging to the two distributions (images and semantic masks) can be effortlessly transferred reversibly.

For semantic segmentation, our approach solves the contradiction between the randomness of diffusion outputs and the uniqueness of segmentation results.

For image synthesis, we propose a finite perturbation approach to enable multi-modal generation and improve the quality of synthesis results.

Visualization

Semantic Segmentation

Semantic Image Synthesis

Citation

If you find this work useful for your research, please consider citing our paper:

@article{wang2024semflow,
  author = {Wang, Chaoyang and Li, Xiangtai and Qi, Lu and Ding, Henghui and Tong, Yunhai and Yang, Ming-Hsuan},
  title = {SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow},
  journal = {arXiv preprint arXiv:2405.20282},
  year = {2024}
}

License

MIT license