Ice Segmentation using Deep Learning

This repository holds the various findings and experimentations that went into creating a preliminary Machine Learning model for Ice Segmentation using Deep Learning methods for the regions affected by this situation in the Inuit-Nunavik Region.

With the help of the CAIMAN network we had over 500 images at our disposal initially to work with and develop a model that would work similar, if not better, than pre-existing models for the same task found during the literature review.

Let us go through the key steps in creating the dataset to work upon and the experiments supporting them.

Initial Experiments on Existing Models

For the initial part of the project, we looked upon recreating results from pre-exisitng models to gauge their own performance on their own dataset. The recreation was done using the publically available repositories.

The key paper that caught our eye was the Supplementing Remote Sensing of Ice: Deep Learning-Based Image Segmentation System for Automatic Detection and Localization of Sea-ice Formations From Close-Range Optical Images which used an ensemble model combining architectures like PSPNet, PSPDenseNet, DeepLabV3+, and UPerNet, to get its result on 450+ highly annotated images.

Using the similar premise, we decided to try to work on an ensemble of pre-known somewhat lightweighed models like PSPNet, U-Net and Mask R-CNN.

Dealing with the Data

However, there was an abundance of data coming from the CAIMAN network, the job of manually annotating the images was a bit tricky given the weather conditions the images were taken in.

I took upon annotating over a 100 images manually using the Pixel Annotation Tool which is easily buildable on both LINUX and Windows.

The annotation technique uses Watershedding feature from OpenCV. Instead of the traditional Thresholding or Contour detection, watershedding performs a marker-based image segmentation on its own depending on neighbor pixels.

Once the images were annotated, the images needed to be renamed and cropped due to their origins from different locations and types of cameras. (code to which can be found in the Utils Folder).

Training the Models

Before training the individual models on the desired dataset, I decided to go for a transfer learning method wherein, I aimed to get the weights from training the models on bigger datasets like ImageNet and the Caravan Dataset and then use them for Ice Segmentation on the Custom Dataset.

U-Net

U-Net is a CNN architecture designed specifically to deal with image segmentation in Bio Medical Images. It can be used to highlight the points of interest all the while semantically segmenting each pixel to the desired class.

It usually is a pre-trained classification network like VGG/ResNet where you apply convolution blocks followed by a maxpool downsampling to encode the input image into feature representations at multiple different levels. The decoder is the second half of the architecture.

PSP Net

PSPNet, or Pyramid Scene Parsing Network, is a semantic segmentation model that utilises a pyramid parsing module that exploits global context information by different-region based context aggregation. The local and global clues together make the final prediction more reliable.

Given an input image, PSPNet use a pretrained CNN with the dilated network strategy to extract the feature map.

Steps to run the Repository

Download the repository using the command:
git clone https://github.com/aryankargwal/ice_segmentation.git

Download the Requirements to run the code using the command:
pip install -r requirements.txt

TODO

License

This project is under the Apache License. See LICENSE for more information.