MultitaskAIS

TensorFlow implementation of the model proposed in "A Multi-Task Deep Learning Architecture for Maritime Surveillance Using AIS Data Streams" (https://ieeexplore.ieee.org/abstract/document/8631498).

All the codes related to the Embedding block are adapted from the source code of Filtering Variational Objectives: https://github.com/tensorflow/models/tree/master/research/fivo

Directory Structure

The elements of the code are organized as follows:

multitaskAIS.py                   # script to run the model (except the Contrario detection)
runners.py                        # graph construction code for training and evaluation
bounds.py                         # code for computing each bound
contrario.py                      # script to run the Contrario detection block
contrario_utils.py
distribution_utils.py
nested_utils.py
utils.py
get_coastline_streetmap.py        # script to download the coastline shapefile
data
├── datasets.py                   # readers for AIS dataset
├── calculate_AIS_mean.py         # calculates the mean of AIS "four-hot" vectors
├── dataset_preprocessing.py      # preprocesses the AIS datasets
└── csv2pkl.py                    # loads AIS data from *.csv files 
models
└── vrnn.py                       # variational RNN implementation
chkpt
└── ...                           # directory to keep checkpoints and summaries in
results
└── ...                           # directory to save outcomes

Requirements:

See requirements.yml

Datasets:

The MarineC dataset is provided by MarineCadastre.gov, Bureau of Ocean Energy Management, and National Oceanic and Atmospheric Administration, (marinecadastre.gov), and availble at (https://marinecadastre.gov/ais/)

The Brittany dataset is provided by CLS-Collecte Localisation Satellites (https://www.cls.fr/en/) and Erwan Guegueniat, contains AIS messages captured by a coastal receiving station in Ushant, from 07/2011 to 01/2018. We provide here a set of preprocessed AIS messages (data/dataset8.zip) on which readers can re-produce the results in the paper. This set contains dynamic information of AIS tracks (LAT, LON, SOG, COG, HEADING, ROT, NAV_STT, TIMESTAMP, MMSI) from 01/2017 to 03/2017, downsampled to a resolution of 5 minutes. For the full Brittany dataset, please contact CLS (G.Hajduch, ghajduch@groupcls.com).

Preprocess the Data

Converting to csv:

MarineC dataset: we use QGIS (https://qgis.org/en/site/) to convert the original metadata format to csv files.
Brittany dataset: we use libais (https://github.com/schwehr/libais) to decode raw AIS messages to csv files.

csv2pkl.py then loads the data from csv files, selects AIS messages in the pre-defined ROI then saves them as pickle format.

Preprocessing steps: the data then processed as discribed in the paper by dataset_preprocessing.py

Training the Embedding layer

First we must train the Embedding layer:

python multitaskAIS.py \
  --mode=train \
  --logdir=./chkpt \
  --bound=elbo \
  --summarize_every=100 \
  --latent_size=100 \
  --batch_size=50 \
  --num_samples=16 \
  --learning_rate=0.0003 \

Running task-specific submodels

After the Embedding layer is trained, we can run task-specific blocks.

save_outcomes

To avoid re-caculating the for each task, we calculate them once and save as a .pkl file.

python multitaskAIS.py \
  --mode=save_outcomes \
  --logdir=./chkpt \
  --trainingset_name=dataset8/dataset8_train.pkl \
  --testset_name=dataset8/dataset8_valid.pkl \
  --bound=elbo \
  --latent_size=100 \
  --batch_size=1 \
  --num_samples=16 \

Similarly for the test set (testset_name=dataset8/dataset8_valid.pkl).

log_density

log_density calculates the distribution of in each small cells of the ROI.

python multitaskAIS.py \
  --mode=log_density \
  --logdir=./chkpt \
  --trainingset_name=dataset8/dataset8_train.pkl \
  --testset_name=dataset8/dataset8_valid.pkl \
  --bound=elbo \
  --latent_size=100 \
  --batch_size=1 \
  --num_samples=16 \

contrario detection

contrario.py performs the contrario detection and plots the results.

python contrario.py \

traj_reconstruction

traj_reconstruction performs the trajectory reconstruction.

python multitaskAIS.py \
  --mode=traj_reconstruction \
  --logdir=./chkpt \
  --trainingset_name=dataset8/dataset8_train.pkl \
  --testset_name=dataset8/dataset8_test.pkl \
  --bound=elbo \
  --latent_size=100 \
  --batch_size=1 \
  --num_samples=16 \

Acknowledgement

We would like to thank MarineCadastre, CLS and Erwan Guegueniat, Tensorflow team and OpenStreetmap for the data and the open-source code.

Contact

This code is a raw version of MultitaskAIS. We are sorry for not providing a clean version of the code, it is being optimized. For any questions/issues, please contact Duong NGUYEN via van.nguyen1@imt-atlantique.fr

leilin-research/MultitaskAIS