/clothing-segmentation

Clothing segmentation with DeepLabV3+

Primary LanguageJupyter NotebookMIT LicenseMIT

Clothes Segmentation using DeepLabV3+

Train the DeepLabV3+ model on Colab with Tensorflow API.

tf

Table of Contents

Introduction

Recently, the fashion industry has undergone a digital transformation by expanding on online platforms. Especially after the pandemic, online platforms have grown considerably commercially and as an area where data scientists can develop applications and provide benefits. This growth allows the development of compelling use cases of artificial intelligence in the fashion industry. For example, extracting clothing products and their related features from pictures can improve consumers' shopping experience and improve work efficiency for fashion workers. Therefore, the automatic labelling and visual analysis of clothing images in this area have attracted increasing attention.

example-result

Training Details

example-result

The DeepLab architecture has proven to be quite successful. This study used a pre-trained inceptionresnetv2 backbone network to parse clothing images into five main categories. 13520 images from iMaterialists (Fashion) 2020 FGVC7 for training have been resized to 256x256. In the study, to reduce the complex structure of the data set, 46 different categories were converted into 5 main categories ( 'upper-body', 'lower-body', 'whole-body', 'feet' and 'accessories'). The Adam optimization method and categorical cross-entropy loss were used for model optimization.

This project uses DeepLabV3 as a deep learning model. However, instead of having 1 channel output from DeepLabV3 for the typical noticeable semantic segmentation task, it outputs 5 channels of output, each representing 5 main categories.

Jupyter Notebook

You can train the DeepLabV3+ model. Click the button below to open it on Colab.

Open In Colab

example-result

Data

Dataset Taxonomy

+---------+-----------------------------------------+----------------+
| ClassId |                   Name                  | SuperCategory  |
+---------+-----------------------------------------+----------------+
|    0    |              shirt, blouse              |   upperbody    |
|    1    |         top, t-shirt, sweatshirt        |   upperbody    |
|    2    |                 sweater                 |   upperbody    |
|    3    |                 cardigan                |   upperbody    |
|    4    |                  jacket                 |   upperbody    |
|    5    |                   vest                  |   upperbody    |
|    6    |                  pants                  |   lowerbody    |
|    7    |                  shorts                 |   lowerbody    |
|    8    |                  skirt                  |   lowerbody    |
|    9    |                   coat                  |   wholebody    |
|    10   |                  dress                  |   wholebody    |
|    11   |                 jumpsuit                |   wholebody    |
|    12   |                   cape                  |   wholebody    |
|    13   |                 glasses                 |      head      |
|    14   |                   hat                   |      head      |
|    15   | headband, head covering, hair accessory |      head      |
|    16   |                   tie                   |      neck      |
|    17   |                  glove                  | arms and hands |
|    18   |                  watch                  | arms and hands |
|    19   |                   belt                  |     waist      |
|    20   |                leg warmer               | legs and feet  |
|    21   |            tights, stockings            | legs and feet  |
|    22   |                   sock                  | legs and feet  |
|    23   |                   shoe                  | legs and feet  |
|    24   |               bag, wallet               |     others     |
|    25   |                  scarf                  |     others     |
|    26   |                 umbrella                |     others     |
|    27   |                   hood                  | garment parts  |
|    28   |                  collar                 | garment parts  |
|    29   |                  lapel                  | garment parts  |
|    30   |                epaulette                | garment parts  |
|    31   |                  sleeve                 | garment parts  |
|    32   |                  pocket                 | garment parts  |
|    33   |                 neckline                | garment parts  |
|    34   |                  buckle                 |    closures    |
|    35   |                  zipper                 |    closures    |
|    36   |                 applique                |  decorations   |
|    37   |                   bead                  |  decorations   |
|    38   |                   bow                   |  decorations   |
|    39   |                  flower                 |  decorations   |
|    40   |                  fringe                 |  decorations   |
|    41   |                  ribbon                 |  decorations   |
|    42   |                  rivet                  |  decorations   |
|    43   |                  ruffle                 |  decorations   |
|    44   |                  sequin                 |  decorations   |
|    45   |                  tassel                 |  decorations   |
+---------+-----------------------------------------+----------------+

More detailed explanation here

Test Results

Here are some visual and numerical results from the iMaterialist test dataset. Reasonably good results have been achieved in this dataset, which is complex, and labels are inconsistent.

loss Avg. IoU (%) F1-Score (%)
0.3414 54.69 61.92

Visual Results

example-result example-result example-result example-result example-result

References