Flower mapping final project for CS 153 at Harvey Mudd College.
The following files can be downloaded. We have included the annotations we used, along with their corresponding masked images. Note that the code in this repository expects all of the data to be present in order to generate the tiles used for training (see Setup below). In theory, the code could be adjusted to generate everything using these masked images, but in order to be more general to other datasets and annotations, we left it expecting all the data.
The weights for our trained models (see the training and running model sections below) can be downloaded here.
Follow the steps below to set up the repository.
-
Install Pipenv.
-
Run
pipenv install
from the root of this repository to install Python dependencies. -
Run
./download-weights.sh
to download the COCO-Stuff DeepLab weights from kazuto1011/deeplab-pytorch. -
Place drone images and JSON data in subdirectories in the
data
directory. For access to the data, contact the HMC Bee Lab.Data directory file structure
Our
data
directory had the following file structure:070921_North 070921_North100_0007_0044.JPG 070921_North100_0007_0044.json ... 071121_CentralEastern 071121_CentralEastern100_0009_0002.JPG 071121_CentralEastern100_0009_0002.json ... 071121_Western 071121_Western100_0011_0015.JPG 071121_Western100_0011_0015.json ... 2017_6617East1 2017_6617East1DJI_0029.JPG 2017_6617East1DJI_0029.json ... 2017_6617East2 2017_6617East2DJI_0345.JPG 2017_6617East2DJI_0345.json ... 2017_6917West 2017_6917WestDJI_0007.JPG 2017_6917WestDJI_0007.json ...
JSON file format
The JSON files associated with each plant are expected to at least have the following data:
{ "classes": [ "ERFA" ], "labels": [ { "class": "ERFA", "segment": [ 1741.56, 1014.43, 1710.62, 1001.17, 1681.89, 981.28, ... ] }, ... ] }
The classes are each type of plant present, and the segment contains the points of a boundary polygon for that plant in the format
[x1, y1, x2, y2, ...]
. -
Run
pipenv run python src/generate_plants.py
to generate plant images in the directoryplant_images
. -
Place ground truth annotation masks for each plant in the
annotations
directory. Note that the dimensions of each annotations should be equal to the bounding box of the polygon created using the OpenCVfillPoly
function for the plant segmentation boundary from the JSON data in step 4. -
Run
pipenv run python src/processing.py
to clean up the annotations and set everything up for the training and running the models.
These steps assume that everything was set up from the previous steps.
-
Run
pipevn run python src/hsv_histogram.py
to generate HSV histograms of flower and non-flower areas across all annotated plants. -
Run
pipenv run python src/hsv.py
to generate HSV histograms of flower and non-flower areas for all annotated plants in theprocessed/hsv_masks
directory.
-
Run
pipenv run python src/xsede-setup.py
to copy the relevant files to thexsede_deeplabv2
andxsede_deeplabv3
directories and copy these directories to XSEDE. Also, change the paths within themodel.job
files to reflect the path on XSEDE. -
On XSEDE,
cd
into each folder and runsbatch model.job
to train the model. Runtail -f output.txt
to see what is going on as the model trains. -
When complete, copy the file
deeplabv2.pth
orweights.pt
(for the DeepLab v3 model) back to your computer and place in themodel
directory.
-
Run
pipenv run python src/run_deeplabv2.py
to generate annotations to theprocessed/deeplabv2_masks
folder, and relevant performance statistics. -
Run
pipenv run python src/run_deeplabv3.py
to generate annotations to theprocessed/deeplabv3_masks
folder, and relevant performance statistics.