Rahul2991/trimap_generator

Generating automatic trimap through pixel dilation and strongly-connected-component algorithms

Automatic Trimap Generator

Keywords: Alpha Compositioning,Trimap

關鍵: Alpha合成、三分圖

キーワード:アルファチャンネル、マスク画像

Introduction

• In image matting, trimap estimates foreground from background by inscribing unknown region
• Mathematically, an image can be represented by the following equation:

In this equation, I_p denotes the entire image, F_p denotes a definite foreground, and B_p denotes a definite background.
On the other hand, is an alpha matte constant with a value ranging between 0 and 1. An value of 0 indicates that the pixel belongs to a background; whereas an value of 1 indicates otherwise. Any value in between means a mixed pixel that has to be determined.

Descriptions

• Generate a trimap (foreground, background, and unknown regions) from an input of binary (mask) image
• Foreground has a pixel value of 255; background has a pixel value of 0; and unknown has a pixel value of 127
• In this example, the trimap is generated by extending a binary image of a previously segmented tumor
• A binary image consists of two parts: foreground (white) which is the tumor and background (black) which is the surrounding region
• Keep in mind that the unknown region is simply an approximation rather than an exact delineation. Therefore, matting process becomes a crucial key to extract foreground images with exact precision
• All binary images in this repository were generated from U-Net with a Jaccard's Coefficient of 0.94 (out of 1), indicating a high agreement between the ground truth and segmented mask images
• Image erosion feature is added which may anticipate any overestimating foreground issue

Input: a binary image (from a segmented lesion)
Output: a trimap with unknown region (gray) from tumor dilation

Flow Chart

---

TO DO:

• Finding The Most Dominant Foreground -- automatic kernel design (generate an odd-sized matrix from an integer)
• Contour Detection Module -- an alternative method to circumscribe foreground without U-Net segmentation
• Mask Contour Module -- create a binary mask from contour point polygon

Examples

1 Dilating the binary image (trimap_module.py)

import cv2, os, sys
from trimap_module import trimap

path    = "./image/samples/seg_image.png";
image   = extractImage(path);
name    = "testImage";
size    = 10; # how many pixel extension do you want to dilate
number  = 1;  # numbering purpose 
trimap(image, name, size, number, erosion=False)

FULL IMAGE	MASK IMAGE	FOREGROUND	BACKGROUND

BINARY IMAGE	TRIMAP (10 PX)	TRIMAP (20 PX)	TRIMAP (30 PX)

2 Handling Non-Dominant Foreground (trimap_module.py)

import cv2, os, sys
from trimap_module import trimap
path    = "./image/test_images/test_image_10.png";
image   = extractImage(path);

kernel  = np.ones( (9,9), np.uint8 ); 
opening = unit01.morph_close(image,kernel);
trimap(opening, "trimap_result", 10, 1, erosion=False)

NOISES	ORIGINAL IMAGES	TRIMAPS PREVIOUS	TRIMAPS NEW
Outside FG
Inside FG

3 Impact of Eroding or Expanding Foreground (trimap_class.py)

import cv2, os, sys
from trimap_class import trimap
path    = "./image/test_images/test_image_12.png";
image   = extractImage(path);
    
trimap(image, ""trimap_result, 10, 1, DEFG=Erosion, num_iter=3);

The illustration starts with zero erosion/dilation; followed with one, three, five, until eleven iterations (an increment of two).

Erosion	Expansion

4 Image Processing using Feature Extraction Module (feature_extraction.py)
Coming Soon

References

1. Vikas Gupta and Shanmuganathan Raman. (2017). "Automatic Trimap Generation for Image Matting". Indian Institute of Technology, Gandhinagar, IND download
2. Olivier Juan and Reanud Keriven. (2005). "Trimap Segmentation for Fast and User-Friendly Alpha Matting". FRA download
3. Jimei Yang, Brian Price, Scott Cohen, Honglak Lee, and Ming-Hsuan Yang. (2016). "Object Contour Detection with a Fully Convolutional Encoder-Decoder Network". Adobe Research, USA download