Deep Automatic Natural Image Matting [IJCAI-21]

This is the official repository of the paper Deep Automatic Natural Image Matting.

Introduction | Network | AIM-500 | Results | Statement

📆 News
The training code, inference code and the pretrained models will be released soon.
[2021-07-16]: Publish the validation dataset AIM-500. Please follow the readme.txt for details.

Introduction

Different from previous methods only focusing on images with salient opaque foregrounds such as humans and animals, in this paper, we investigate the difficulties when extending the automatic matting methods to natural images with salient transparent/meticulous foregrounds or non-salient foregrounds.

To address the problem, we propose a novel end-to-end matting network, which can predict a generalized trimap for any image of the above types as a unified semantic representation. Simultaneously, the learned semantic features guide the matting network to focus on the transition areas via an attention mechanism.

We also construct a test set AIM-500 that contains 500 diverse natural images covering all types along with manually labeled alpha mattes, making it feasible to benchmark the generalization ability of AIM models. Results of the experiments demonstrate that our network trained on available composite matting datasets outperforms existing methods both objectively and subjectively.

Network

We propose the methods consist of:

Improved Backbone for Matting: an advanced max-pooling version of ResNet-34, serves as the backbone for the matting network, pretrained on ImageNet;
Unified Semantic Representation: a type-wise semantic representation to replace the traditional trimaps;
Guided Matting Process: an attention based mechanism to guide the matting process by leveraging the learned semantic features from the semantic decoder to focus on extracting details only within transition area.

The backbone pretrained on ImageNet and the model pretrained on synthetic matting dataset will be released soon.

Pretrained-backbone	Pretrained-model
coming soon	coming soon

AIM-500

We propose AIM-500 (Automatic Image Matting-500), the first natural image matting test set, which contains 500 high-resolution real-world natural images from all three types (SO, STM, NS), many categories, and the manually labeled alpha mattes. Some examples and the amount of each category are shown below. The AIM-500 dataset is published now, can be downloaded directly from this link. Please follow the readme.txt for more details.

Portrait	Animal	Transparent	Plant	Furniture	Toy	Fruit
100	200	34	75	45	36	10

Results

We test our network on different types of images in AIM-500 and compare with previous SOTA methods, the results are shown below.

Statement

If you are interested in our work, please consider citing the following:

@inproceedings{ijcai2021-danim,
  title     = {Deep Automatic Natural Image Matting},
  author    = {Li, Jizhizi and Zhang, Jing and Tao, Dacheng},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  year      = {2021},
}

This project is under the MIT license. For further questions, please contact jili8515@uni.sydney.edu.au.

Relevant Projects

End-to-end Animal Image Matting
Jizhizi Li, Jing Zhang, Stephen J. Maybank, Dacheng Tao

xueyedamo521/AIM