/Awesome-Image-Inpainting

Awesome-Inpainting-Tech Awesome

A curated list of inpainting papers and resources, inspired by awesome-computer-vision.

Contents

teaser

Image Inpainting

Classical methods (Non-learning based)

  1. Image inpainting. Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. SIGGRAPH2000.
  2. Simultaneous structure and texture image inpainting. Bertalmio, M., Vese, L., Sapiro, G., & Osher, S. TIP2003.
  3. Region filling and object removal by exemplar-based image inpainting. Criminisi, A., Pérez, P., & Toyama, K. TIP2004.
  4. Image completion with structure propagation. Sun, J., Yuan, L., Jia, J., & Shum, H. Y. TOG2005.
  5. Image completion using planar structure guidance. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. TOG2014. [code] [project]

Deep Architectures (Learning Based)

  1. Shepard convolutional neural networks. Ren, J. S., Xu, L., Yan, Q., & Sun, W. NeurIPS2015. [code]
  2. Context encoders: Feature learning by inpainting. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. CVPR2016. [code]
  3. Globally and locally consistent image completion. Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017). TOG2017. [code] [project]
  4. High-resolution image inpainting using multi-scale neural patch synthesis. Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. CVPR2017. [code]
  5. Generative face completion. Li, Y., Liu, S., Yang, J., & Yang, M. H. CVPR2017. [code]
  6. Semantic image inpainting with deep generative models. Yeh, R. A., Chen, C., Yian Lim, T., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. CVPR2017. [code] [project]
  7. Generative image inpainting with contextual attention. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. CVPR2018. [code] [project]
  8. Natural and effective obfuscation by head inpainting. Sun Qianru et al. CVPR2018.
  9. Eye in-painting with exemplar generative adversarial networks. Dolhansky, B., & Canton Ferrer, C. CVPR2018. [project] [code]
  10. Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition. Deng, J., Cheng, S., Xue, N., Zhou, Y., & Zafeiriou, S. CVPR2018.
  11. Disentangling Structure and Aesthetics for Style-aware Image Completion. Gilbert, A., Collomosse, J., Jin, H., & Price, B. CVPR2018.
  12. Image inpainting for irregular holes using partial convolutions. Liu, G., Reda, F. A., Shih, K. J., Wang, T. C., Tao, A., & Catanzaro, B. ECCV2018. [project]
  13. Contextual-based image inpainting: Infer, match, and translate. Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Jay Kuo, C. C. ECCV2018.
  14. Shift-net: Image inpainting via deep feature rearrangement. Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. ECCV2018. [code]
  15. Image Inpainting via Generative Multi-column Convolutional Neural Networks. Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. NeurIPS2018. [code]
  16. SPG-Net: Segmentation prediction and guidance network for image inpainting. Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. C. J. BMVC2018.
  17. Structural inpainting. Vo, H. V., Duong, N. Q., & Pérez, P. MM2018.
  18. Semantic Image Inpainting with Progressive Generative Networks. Zhang, H., Hu, Z., Luo, C., Zuo, W., & Wang, M. MM2018. [code]
  19. Face Completion with Semantic Knowledge and Collaborative Adversarial Learning. Liao, H., Funka-Lea, G., Zheng, Y., Luo, J., & Zhou, S. K. ACCV2018.
  20. Edge-Aware Context Encoder for Image Inpainting. Liao, L., Hu, R., Xiao, J., & Wang, Z. ICASPP2018.
  21. Faceshop: Deep sketch-based face image editing. Portenier, T., Hu, Q., Szabó, A., Bigdeli, S. A., Favaro, P., & Zwicker, M. TOG2018.
  22. High resolution face completion with multiple controllable attributes via fully end-to-end progressive generative adversarial networks. Chen, Z., Nie, S., Wu, T., & Healey, C. G. Arxiv2018.
  23. On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs. Banerjee, S., Scheirer, W. J., Bowyer, K. W., & Flynn, P. J. Arxiv2018.
  24. Free-form image inpainting with gated convolution. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. Arxiv2018. [project]
  25. Pluralistic Image Completion. Zheng, C., Cham, T. J., & Cai, J. CVPR2019. [code] [project]
  26. Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting. Zeng, Y., Fu, J., Chao, H., & Guo, B. CVPR2019. [code]
  27. Foreground-aware Image Inpainting. Xiong, W., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. CVPR2019.
  28. Deep Reinforcement Learning of Volume-guided Progressive View Inpainting for 3D Point Scene Completion from a Single Depth Image. Han, X., Zhang, Z., Du, D., Yang, M., Yu, J., Pan, P., ... & Cui, S. CVPR2019.
  29. PEPSI: Fast Image Inpainting With Parallel Decoding Network. CVPR (pp. 11360-11368). Sagong, M. C., Shin, Y. G., Kim, S. W., Park, S., & Ko, S. J. CVPR2019.
  30. Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation. Grigorev, A., Sevastopolsky, A., Vakhitov, A., & Lempitsky, V. CVPR2019.
  31. EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning. Nazeri, K., Ng, E., Joseph, T., Qureshi, F., & Ebrahimi, M. Arxiv2019. [code]
  32. Deep Inception Generative Network for Cognitive Image Inpainting. Xiao, Q., Li, G., & Chen, Q. Arxiv2019.
  33. Detecting Overfitting of Deep Generative Networks via Latent Recovery. Webster, R., Rabin, J., Simon, L., & Jurie, F. Arxiv2019.
  34. SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color. Jo, Y., & Park, J. (2019). Arxiv2019. [code]
  35. Deep Fusion Network for Image Completion. Hong, X., Xiong, P., Ji, R., & Fan, H. Arxiv2019. [code]
  36. PEPSI++: Fast and Lightweight Network for Image Inpainting. Shin, Y. G., Sagong, M. C., Yeo, Y. J., Kim, S. W., & Ko, S. J. Arxiv2019.
  37. Generative Image Inpainting with Submanifold Alignment Ang Li, Jianzhong Qi, Rui Zhang, Xingjun Ma, Kotagiri Ramamohanarao. In IJCAI2019.

Video Inpainting

Classical methods (Non-learning based)

  1. Navier-stokes, fluid dynamics, and image and video inpainting. Bertalmio, M., Bertozzi, A. L., & Sapiro, G. CVPR2001 (Vol. 1, pp. I-I). IEEE.
  2. Video inpainting of occluding and occluded objects. Patwardhan, K. A., Sapiro, G., & Bertalmio, M. In IEEE International Conference on Image Processing 2005 (Vol. 2, pp. II-69). IEEE.
  3. Full-frame video stabilization with motion inpainting. Matsushita, Y., Ofek, E., Ge, W., Tang, X., & Shum, H. Y. TPAMI2006, (7), 1150-1163. [project]
  4. Video completion by motion field transfer. Shiratori, T., Matsushita, Y., Tang, X., & Kang, S. B. (2006, June). CVPR2006 (Vol. 1, pp. 411-418). [project]
  5. Space-time completion of video. Wexler, Y., Shechtman, E., & Irani, M. TPAMI2007, (3), 463-476. [project]
  6. Video inpainting under constrained camera motion. Patwardhan, K. A., Sapiro, G., & Bertalmío, M. TIP2007, 16(2), 545-553.
  7. How not to be seen—object removal from videos of crowded scenes. Granados, M., Tompkin, J., Kim, K., Grau, O., Kautz, J., & Theobalt, C. In Computer Graphics Forum 2012 (Vol. 31, No. 2pt1, pp. 219-228). [project]
  8. Background inpainting for videos with dynamic objects and a free-moving camera. Springer, Berlin, Heidelberg. Granados, M., Kim, K. I., Tompkin, J., Kautz, J., & Theobalt, C. ECCV2012. [project]
  9. Video inpainting of complex scenes. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., & Pérez, P. SIAM Journal on Imaging Sciences 2014, 7(4), 1993-2019. [project]
  10. Video inpainting with short-term windows: application to object removal and error concealment. Ebdelli, M., Le Meur, O., & Guillemot, C. TIP2015, 24(10), 3034-3047.
  11. Temporally coherent completion of dynamic video. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. TOG2016. [project] [code]

Deep Architectures (Learning Based)

  1. Video inpainting by jointly learning temporal structure and spatial details. Wang, C., Huang, H., Han, X., & Wang, J. AAAI2019.
  2. Deep Flow-Guided Video Inpainting. Xu, R., Li, X., Zhou, B., & Loy, C. C. CVPR2019. [code] [project]
  3. Deep Video Inpainting. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. CVPR2019. [code] [project]
  4. Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. CVPR2019. [project]
  5. VORNet: Spatio-temporally Consistent Video Inpainting for Object Removal Ya-Liang Chang, Zhe Yu Liu, Winston Hsu. In CVPRW2019. [code]
  6. Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, Winston Hsu. In ICCV2019. [code]
  7. Learnable Gated Temporal Shift Module for Deep Video Inpainting Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, Winston Hsu. In BMVC2019. [code]
  8. Align-and-Attend Network for Globally and Locally Coherent Video Inpainting. Woo, S., Kim, D., Park, K., Lee, J. Y., & Kweon, I. S. Arxiv2019.
  9. Frame-Recurrent Video Inpainting by Robust Optical Flow Inference Yifan Ding, Chuan Wang, Haibin Huang, Jiaming Liu, Jue Wang, Liqiang Wang. In Arxiv 2019.

Challenge

  1. 2018 Looking at People ECCV Satellite Challenge
  2. 2019 ICME Grand Challenge Learning-Based Image Inpainting