damminhtien/awesome-semantic-segmentation

awesome-semantic-segmentation - list of awesome things around semantic segmentation :tada:

Awesome Semantic Segmentation

List of awesome things around semantic segmentation 🎉

Semantic segmentation is a computer vision task in which we label specific regions of an image according to what's being shown. Semantic segmentation awswers for the question: "What's in this image, and where in the image is it located?".

Semantic segmentation is a critical module in robotics related applications, especially autonomous driving, remote sensing. Most of the research on semantic segmentation is focused on improving the accuracy with less attention paid to computationally efficient solutions.

The recent appoarch in semantic segmentation is using deep neural network, specifically Fully Convolutional Network (a.k.a FCN). We can follow the trend of semantic segmenation approach at: paper-with-code.

Evaluate metrics: mIOU, accuracy, speed,...

State-Of-The-Art (SOTA) methods of Semantic Segmentation

	Paper	Benchmark on PASALVOC12	Release	Implement
EfficientNet-L2+NAS-FPN	Rethinking Pre-training and Self-training	90.5%	NeurIPS 2020	TF
DeepLab V3+	Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation	89%	ECCV 2018	TF, Keras, Pytorch, Demo
DeepLab V3	Rethinking Atrous Convolution for Semantic Image Segmentation	86.9%	17 Jun 2017	TF, TF
Smooth Network with Channel Attention Block	Learning a Discriminative Feature Network for Semantic Segmentation	86.2%	CVPR 2018	Pytorch
PSPNet	Pyramid Scene Parsing Network	85.4%	CVPR 2017	Keras, Pytorch, Pytorch
ResNet-38 MS COCO	Wider or Deeper: Revisiting the ResNet Model for Visual Recognition	84.9%	30 Nov 2016	MXNet
RefineNet	RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation	84.2%	CVPR 2017	Matlab, Keras
GCN	Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network	83.6%	CVPR 2017	TF
CRF-RNN	Conditional Random Fields as Recurrent Neural Networks	74.7%	ICCV 2015	Matlab, TF
ParseNet	ParseNet: Looking Wider to See Better	69.8%	15 Jun 2015	Caffe
Dilated Convolutions	Multi-Scale Context Aggregation by Dilated Convolutions	67.6%	23 Nov 2015	Caffe
FCN	Fully Convolutional Networks for Semantic Segmentation	67.2%	CVPR 2015	Caffe

Variants

• FCN with VGG(Resnet, Densenet) backbone: pytorch
• The easiest implementation of fully convolutional networks (FCN8s VGG): pytorch
• TernausNet (UNet model with VGG11 encoder pre-trained on Kaggle Carvana dataset paper: pytorch
• TernausNetV2: Fully Convolutional Network for Instance Segmentation: pytorch
• Light-Weight RefineNet for Real-Time Semantic Segmentation

Review list of Semantic Segmentation

• A 2021 guide to Semantic Segmentation (nanonet) ⭐ ⭐ ⭐ ⭐
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey 2020 (University of Gour Banga,India) ⭐ ⭐ ⭐ ⭐ ⭐
• A peek of Semantic Segmentation 2018 (mc.ai) ⭐ ⭐ ⭐ ⭐
• Semantic Segmentation guide 2018 (towardds) ⭐ ⭐ ⭐ ⭐
• An overview of semantic image segmentation (jeremyjordan.me) ⭐ ⭐ ⭐ ⭐ ⭐
• Recent progress in semantic image segmentation 2018 (arxiv, towardsdatascience) ⭐ ⭐ ⭐ ⭐
• A 2017 Guide to Semantic Segmentation Deep Learning Review (blog.qure.ai) ⭐ ⭐ ⭐ ⭐ ⭐
• Review popular network architecture (medium-towardds) ⭐ ⭐ ⭐ ⭐ ⭐
• Lecture 11 - Detection and Segmentation - CS231n (slide, vid): ⭐ ⭐ ⭐ ⭐ ⭐
• A Survey of Semantic Segmentation 2016 (arxiv) ⭐ ⭐ ⭐ ⭐ ⭐

Case studies

• Dstl Satellite Imagery Competition, 3rd Place Winners' Interview: Vladimir & Sergey: Blog, Code
• Carvana Image Masking Challenge–1st Place Winner's Interview: Blog, Code
• Data Science Bowl 2017, Predicting Lung Cancer: Solution Write-up, Team Deep Breath: Blog
• MICCAI 2017 Robotic Instrument Segmentation: Code and explain
• 2018 Data Science Bowl Find the nuclei in divergent images to advance medical discovery: 1st place, 2nd, 3rd, 4th, 5th, 10th
• Airbus Ship Detection Challenge: 4th place, 6th
• iMaterialist (Fashion) 2020 at FGVC7: 1st place
• Understanding Clouds from Satellite Images: 1st place, 2nd, 3rd
• Global Wheat Detection: 1st place, 2nd, 3rd
• Severstal: Steel Defect Detection: 1st place, 4th, 7th
• Human Protein Atlas Image Classification: 1st place, 5th

Most used loss functions

• Pixel-wise cross entropy loss:
• Dice loss: which is pretty nice for balancing dataset
• Focal loss:
• Lovasz-Softmax loss:

Datasets

• Visual Object Classes Challenge 2012 (VOC2012): 400+ classes of real-world data
• COCO Dataset: 164k images, 72 classes: 80 thing classes, 91 stuff classes and 1 class 'unlabeled'
• Cityscapes: This dataset consists of segmentation ground truths for roads, lanes, vehicles and objects on road. The dataset contains 30 classes and of 50 cities collected over different environmental and weather conditions
• PASCAL-Context
• ADE20K: 20k+ images
• Semantic3d
• CamVid
• lartpang/awesome-segmentation-saliency-dataset
• Kaggle

Frameworks for segmentation

• Semantic Segmentation in PyTorch (by yassouali): Semantic segmentation models, datasets and losses implemented in PyTorch.
• Semantic Segmentation Suite (by George Seif): Semantic Segmentation Suite in TensorFlow. Implement, train, and test new Semantic Segmentation models easily!
• Segmentation Training Pipeline: Research Pipeline for image masking/segmentation in Keras
• Tramac/awesome-semantic-segmentation-pytorch Semantic Segmentation on PyTorch (include FCN, PSPNet, Deeplabv3, Deeplabv3+, DANet, DenseASPP, BiSeNet, EncNet, DUNet, ICNet, ENet, OCNet, CCNet, PSANet, CGNet, ESPNet, LEDNet, DFANet)
• CSAILVision/semantic-segmentation-pytorch Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset
• divamgupta/image-segmentation-keras Implementation of Segnet, FCN, UNet , PSPNet and other models in Keras.
• PaddlePaddle/PaddleSeg: Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc. paper

Related techniques

• Atrous/ Dilated Convolution
• Transpose Convolution (Deconvolution, Upconvolution)
• Unpooling
• A technical report on convolution arithmetic in the context of deep learning
• CRF

Feel free to show your ❤️ by giving a star ⭐

🎁 Check Out the List of Contributors - Feel free to add your details here!