This project is about ...

This is our project for our Computer Vision course, in which we preprocess images taken by a drone in a staged search & rescue scenario, and apply EfficientDet to detect people who are (partially) obscured by trees.

The EfficientDet model (code, paper)

# clone this repository and also EfficientDet
git clone --recursive  https://github.com/luk400/computer-vision-project.git

# download pretrained weights into Yet-Another-EfficientDet-Pytorch/weights
python download_pretrained_weights.py

Data

1. Download the original data - it has to be in a folder data/ in this repo

2. Create all the data necessary (images and json) + augmentation for training

# saves it by default under Yet-Another-EfficientDet-Pytorch/datasets/
run matlab preprocess_data.m

3. To start training, after installing the necessary dependencies, you will also need to create the yaml file needed for training (for an example which is used for the coco-dataset see here) inside the projects folder (Yet-Another-EfficientDet-Pytorch/projects/cv_project.yml) and edit it appropriately, in particular, for train.py to run, you will need to include:

touch Yet-Another-EfficientDet-Pytorch/projects/cv_project.yml

# required fields to be specified
project_name: cv_project
train_set: train
eval_set: eval
obj_list: ['person']

Train

python thermal_efficientdet_train.py --project cv_project --compound_coeff 0 --batch_size 8 --weights ./Yet-Another-EfficientDet-Pytorch/weights/efficientdet-d0.pth

Test

python generate_evaluation_json.py --weights PATH_TO_CHECKPOINT

# this outputs the mAP for the json generated above
run matlab evaluate.m to 

Visualize output

python visualize_test.py --weights ./Yet-Another-EfficientDet-Pytorch/logs/cv_project/weights-training-c2/efficientdet-d2_116_29500.pth 

Best Results:

500 epochs on not relabelled data (but augmented) tensorboard.

500 epochs on relabelled and augmented data tensorboard.

Last training on D1 and D2:{ "20210104-204331/Loss/train" resolution c==1, not so good as expected, it overfits - batch size 4

check this tensorboard plot

check also this tensorboard }

coefficient pth_download mAP (61 testset images)
D0 efficientdet-d0_236_22500.pth 0.70
D1 efficientdet-d1_102_19500.pth 0.87
D2 efficientdet-d2_116_29500.pth 0.88

TODO:

  • improve data augmentation
  • git repo refactoring
  • do training for the others D*
  • make a matlab evaluation script for python results json comparison
  • modify bad labels using relabel_data.m
  • remove images with none labels from dataloaders (e.g. instances_train.json)
  • relabeling feature
  • improve image integration
  • fix data augmentation