Awesome-Inpainting-Tech Awesome

A curated list of inpainting papers and resources, inspired by awesome-computer-vision.

Contents

teaser

Image Inpainting

Classical methods (Non-learning based)

  1. Image inpainting. Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. In SIGGRAPH 2000
  2. Simultaneous structure and texture image inpainting. Bertalmio, M., Vese, L., Sapiro, G., & Osher, S. In TIP 2003
  3. Region filling and object removal by exemplar-based image inpainting. Criminisi, A., Pérez, P., & Toyama, K. In TIP 2004
  4. Image completion with structure propagation. Sun, J., Yuan, L., Jia, J., & Shum, H. Y. In TOG 2005
  5. Image completion using planar structure guidance. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. In TOG 2014 [code] [project]

Deep Architectures (Learning Based)

  1. Shepard convolutional neural networks. Ren, J. S., Xu, L., Yan, Q., & Sun, W. In NeurIPS 2015 [code]
  2. Context encoders: Feature learning by inpainting. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. In CVPR 2016. [code]
  3. Globally and locally consistent image completion. Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017). In TOG 2017 [code] [project]
  4. High-resolution image inpainting using multi-scale neural patch synthesis. Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. In CVPR 2017 [code]
  5. Generative face completion. Li, Y., Liu, S., Yang, J., & Yang, M. H. In CVPR 2017 [code]
  6. Semantic image inpainting with deep generative models. Yeh, R. A., Chen, C., Yian Lim, T., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. In CVPR 2017 [code] [project]
  7. Generative image inpainting with contextual attention. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. In CVPR 2018 [code] [project]
  8. Natural and effective obfuscation by head inpainting. Sun Qianru et al. In CVPR 2018
  9. Eye in-painting with exemplar generative adversarial networks. Dolhansky, B., & Canton Ferrer, C. In CVPR 2018 [project] [code]
  10. Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition. Deng, J., Cheng, S., Xue, N., Zhou, Y., & Zafeiriou, S. In CVPR 2018
  11. Disentangling Structure and Aesthetics for Style-aware Image Completion. Gilbert, A., Collomosse, J., Jin, H., & Price, B. In CVPR 2018
  12. Image inpainting for irregular holes using partial convolutions. Liu, G., Reda, F. A., Shih, K. J., Wang, T. C., Tao, A., & Catanzaro, B. In ECCV 2018 [project]
  13. Contextual-based image inpainting: Infer, match, and translate. Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Jay Kuo, C. C. In ECCV 2018
  14. Shift-net: Image inpainting via deep feature rearrangement. Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. In ECCV 2018 [code]
  15. Image Inpainting via Generative Multi-column Convolutional Neural Networks. Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. In NeurIPS 2018 [code]
  16. SPG-Net: Segmentation prediction and guidance network for image inpainting. Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. C. J. In BMVC 2018
  17. Structural inpainting. Vo, H. V., Duong, N. Q., & Pérez, P. In MM 2018
  18. Semantic Image Inpainting with Progressive Generative Networks. Zhang, H., Hu, Z., Luo, C., Zuo, W., & Wang, M. In MM 2018 [code]
  19. Face Completion with Semantic Knowledge and Collaborative Adversarial Learning. Liao, H., Funka-Lea, G., Zheng, Y., Luo, J., & Zhou, S. K. In ACCV 2018
  20. Edge-Aware Context Encoder for Image Inpainting. Liao, L., Hu, R., Xiao, J., & Wang, Z. In ICASPP 2018
  21. Faceshop: Deep sketch-based face image editing. Portenier, T., Hu, Q., Szabó, A., Bigdeli, S. A., Favaro, P., & Zwicker, M. In TOG 2018
  22. High resolution face completion with multiple controllable attributes via fully end-to-end progressive generative adversarial networks. Chen, Z., Nie, S., Wu, T., & Healey, C. G. In Arxiv 2018
  23. On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs. Banerjee, S., Scheirer, W. J., Bowyer, K. W., & Flynn, P. J. In Arxiv 2018
  24. Pluralistic Image Completion. Zheng, C., Cham, T. J., & Cai, J. In CVPR 2019 [code] [project]
  25. Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting. Zeng, Y., Fu, J., Chao, H., & Guo, B. In CVPR 2019 [code]
  26. Foreground-aware Image Inpainting. Xiong, W., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. In CVPR 2019
  27. Deep Reinforcement Learning of Volume-guided Progressive View Inpainting for 3D Point Scene Completion from a Single Depth Image. Han, X., Zhang, Z., Du, D., Yang, M., Yu, J., Pan, P., ... & Cui, S. In CVPR 2019
  28. PEPSI: Fast Image Inpainting With Parallel Decoding Network. CVPR (pp. 11360-11368). Sagong, M. C., Shin, Y. G., Kim, S. W., Park, S., & Ko, S. J. In CVPR 2019
  29. Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation. Grigorev, A., Sevastopolsky, A., Vakhitov, A., & Lempitsky, V. CVPR2019.
  30. Deep Inception Generative Network for Cognitive Image Inpainting. Xiao, Q., Li, G., & Chen, Q. In Arxiv 2019
  31. Detecting Overfitting of Deep Generative Networks via Latent Recovery. Webster, R., Rabin, J., Simon, L., & Jurie, F. In Arxiv 2019
  32. SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color. Jo, Y., & Park, J. (2019). In Arxiv 2019 [code]
  33. Deep Fusion Network for Image Completion. Hong, X., Xiong, P., Ji, R., & Fan, H. In Arxiv 2019 [code]
  34. PEPSI++: Fast and Lightweight Network for Image Inpainting. Shin, Y. G., Sagong, M. C., Yeo, Y. J., Kim, S. W., & Ko, S. J. In Arxiv 2019
  35. Generative Image Inpainting with Submanifold Alignment Ang Li, Jianzhong Qi, Rui Zhang, Xingjun Ma, Kotagiri Ramamohanarao. In IJCAI 2019
  36. MUSICAL: Multi-Scale Image Contextual Attention Learning for Inpainting. Wang, N., Li, J., Zhang, L., & Du, B. In IJCAI 2019
  37. Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation. Ma, Y., Liu, X., Bai, S., Wang, L., He, D., & Liu, A. In IJCAI 2019
  38. StructureFlow: Image Inpainting via Structure-aware Appearance Flow, Yurui Ren, Xiaoming Yu, Ruonan Zhang, Thomas H. Li, Shan Liu, Ge Li. In ICCV 2019 [code]
  39. Image Inpainting with Learnable Bidirectional Attention Maps, Chaohao Xie, Shaohui Liu, Chao Li, Ming-Ming Cheng, Wangmeng Zuo, Xiao Liu, Shilei Wen, Errui Ding. In ICCV 2019 [code]
  40. Coherent Semantic Attention for Image Inpainting, Hongyu Liu, Bin Jiang, Yi Xiao, Chao Yang. In ICCV 2019 [code]
  41. EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning. Nazeri, K., Ng, E., Joseph, T., Qureshi, F., & Ebrahimi, M. In ICCV 2019 [code]
  42. Free-form image inpainting with gated convolution. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. In ICCV 2019 [project]

Video Inpainting

Classical methods (Non-learning based)

  1. Navier-stokes, fluid dynamics, and image and video inpainting. Bertalmio, M., Bertozzi, A. L., & Sapiro, G. In CVPR 2001 (Vol. 1, pp. I-I). IEEE.
  2. Video inpainting of occluding and occluded objects. Patwardhan, K. A., Sapiro, G., & Bertalmio, M. In ICIP 2005 (Vol. 2, pp. II-69). IEEE.
  3. Full-frame video stabilization with motion inpainting. Matsushita, Y., Ofek, E., Ge, W., Tang, X., & Shum, H. Y. In TPAMI 2006, (7), 1150-1163. [project]
  4. Video completion by motion field transfer. Shiratori, T., Matsushita, Y., Tang, X., & Kang, S. B. (2006, June). In CVPR 2006 (Vol. 1, pp. 411-418). [project]
  5. Space-time completion of video. Wexler, Y., Shechtman, E., & Irani, M. In TPAMI 2007, (3), 463-476. [project]
  6. Video inpainting under constrained camera motion. Patwardhan, K. A., Sapiro, G., & Bertalmío, M. In TIP 2007, 16(2), 545-553.
  7. How not to be seen—object removal from videos of crowded scenes. Granados, M., Tompkin, J., Kim, K., Grau, O., Kautz, J., & Theobalt, C. In Computer Graphics Forum 2012 (Vol. 31, No. 2pt1, pp. 219-228). [project]
  8. Background inpainting for videos with dynamic objects and a free-moving camera. Springer, Berlin, Heidelberg. Granados, M., Kim, K. I., Tompkin, J., Kautz, J., & Theobalt, C. In ECCV 2012. [project]
  9. Video inpainting of complex scenes. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., & Pérez, P. In SIAM Journal on Imaging Sciences 2014, 7(4), 1993-2019. [project]
  10. Video inpainting with short-term windows: application to object removal and error concealment. Ebdelli, M., Le Meur, O., & Guillemot, C. In TIP 2015, 24(10), 3034-3047.
  11. Temporally coherent completion of dynamic video. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. In TOG 2016. [project] [code]

Deep Architectures (Learning Based)

  1. Video inpainting by jointly learning temporal structure and spatial details. Wang, C., Huang, H., Han, X., & Wang, J. In AAAI 2019
  2. Deep Flow-Guided Video Inpainting. Xu, R., Li, X., Zhou, B., & Loy, C. C. In CVPR 2019 [code] [project]
  3. Deep Video Inpainting. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. In CVPR 2019 [code] [project]
  4. Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. In CVPR 2019 [project]
  5. VORNet: Spatio-temporally Consistent Video Inpainting for Object Removal Ya-Liang Chang, Zhe Yu Liu, Winston Hsu. In CVPRW 2019 [code]
  6. Learnable Gated Temporal Shift Module for Deep Video Inpainting Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, Winston Hsu. In BMVC 2019 [code]
  7. Align-and-Attend Network for Globally and Locally Coherent Video Inpainting. Woo, S., Kim, D., Park, K., Lee, J. Y., & Kweon, I. S. In Arxiv 2019
  8. Frame-Recurrent Video Inpainting by Robust Optical Flow Inference Yifan Ding, Chuan Wang, Haibin Huang, Jiaming Liu, Jue Wang, Liqiang Wang. In Arxiv 2019
  9. Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, Winston Hsu. In ICCV 2019 [code]
  10. Onion-Peel Networks for Deep Video Completion, Seoung Wug Oh, Sungho Lee, Joon-Young Lee, Seon Joo Kim. In ICCV 2019 [project] [code]
  11. Copy-and-Paste Networks for Deep Video Inpainting, Sunho Lee, Seoung Wug Oh, DaeYeun Won, Seon Joo Kim. In ICCV 2019 [code][project]
  12. An Internal Learning Approach to Video Inpainting Haotian Zhang, Long Mai, Ning Xu, Zhaowen Wang, John Collomosse, Hailin Jin. In ICCV 2019 [project]

Challenge

  1. 2018 Looking at People ECCV Satellite Challenge
  2. 2019 ICME Grand Challenge Learning-Based Image Inpainting