/instance-segmentation-with-discriminative-loss-tensorflow

Tensorflow implementation of "Semantic Instance Segmentation with a Discriminative Loss Function"

Primary LanguagePythonMIT LicenseMIT

Instance Segmentation with a Discriminative Loss Function

Tensorflow implementation of Semantic Instance Segmentation with a Discriminative Loss Function trained on the TuSimple dataset

Files

├── data here the data should be stored
│        └── tusimple_dataset_processing.py processes the TuSimple dataset
├── doc documentation
├── inference_test inference related data
│        └── images for testing the inference
├── trained_model pretrained model for finetuning
├── clustering.py mean-shift clustering
├── datagenerator.py feeds data for training and evaluation
├── enet.py Enet architecture
├── inference.py tests inference on images
├── loss.py defines discriminative loss function
├── README.md
├── training.py contains training pipeline
├── utils.py contains utilities files for building and initializing the graph
└── visualization.py contains visualization of the clustering and pixel embeddings

Instructions

Inference

  1. To test the inference of the trained model execute:
    python inference.py --modeldir trained_model --outdir inference_test/results

Training

  1. Download the TuSimple training dataset and extract its contents to the data folder. The folder structure should look like this:
    | data
    ├── train_set
    │     ├── clips
    │     ├── label_data_0313.json
    │     ├── label_data_0531.json
    │     ├── label_data_0601.json
    │     └── readme.md
  2. Run the following script to prepare images and labels.
    python data/tusimple_dataset_processing.py <train_data_dir>
    This should create the following images and labels folders:
    | data
    ├── train_set
    ├── images
    └── labels
  3. For training on the dataset execute:
    python training.py
    alternatively use optional parameters (default parameters in this example):
    python training --srcdir data --modeldir pretrained_semantic_model --outdir saved_model --logdir log --epochs 50 --var 1.0 --dist 1.0 --reg 1.0 --dvar 0.5 --ddist 1.5
  4. To test the trained network execute: python inference.py --modeldir saved_model

Training Pipeline

Training Visualization

Feature space projection of one image for consecutive gradient steps. Each point represents one pixel embedding and each color represents an instance in the label.

Results

Todo

  • pip requirements
  • semantic segmentation code
  • visualization

Tensorflow version 1.2

Reference and Credits

This application uses Open Source components. We acknowledge and are grateful to these developers for their contributions to open source: