This is a project for the OpenAI challenge that aimed to detect and localize trees and streets from aerial imagery. The project consist of two CNN model for trees and street respectively. And some code for path optimization, as an example of putting these two parts together. The video of the project is here
All the codes for this project are uploaded. The tree localizer was trained with tensorflow in python, and a prototype application of it was implemented in JAVA. The trained model was also uploaded as a .pb file. The street detector was trained and validated in Mathematica with the reference of Mathematica implementation of SegNet. The rest part (street filtering and path optimization) were also in Mathematica
The training data are not provided with the code as they are just too large to upload. The data for tree localization is from the OpenAI challenge, in the format of raw aerial imagery and the geo-location of all trees. The data for street recognition is from ISPRS commission II/4 dataset.
we are working on a easy-use runable package so that you dont have to bother the following steps anymore
A simple python example is given here(It uses the same .pb file with the java example).
Download the .py file and the data folder, put them together in a new folder, then type in terminal: python tree_detect.py. It will search for all the images in data/test and run prediction for each image, the results will look like this
dependence: tensorflow, numpy, opencv (cv2), matplotlib
Please follow this file for details
The trained model is here in .pb format, which is a serialized tensorflow graph definition with all variables as constants. Loading it restores both the graph definition and the weights. The method on how to operate it can be found here and here
Copy all the files from "localizer_test" folder to your own JAVA project folder, having these packages in your build path:
processing 2.2.1
tensorflow for java
openCV 3.3
apache commons compress
JTS 1.13
The MakePrediction.java makes prediction on a full-size aerial imagery, and export the result as both csv file and equal-sized bounding box image. the csv file shows all the trees found in the imagery in pixel space, as (x, y, type). the rendered image can be overlapped with the input imagery for visualization. Set the variables: root, satelliteImage, predictedCsv, rendering to your input and output file and the model.loadModel() in setup function to where the .pb file is placed. Then launch the program and wait for the result.
String root = "where your imagery is, end with \\";
String satelliteImage = root + "name of input imagery";
String predictedCsv = root + "name of output.csv";
String rendering = root + "name of output.jpg";
model.loadModel("path, end with \\", "name.pb");
The TestTrainedModel_AnyPosition_ForVideo.java is an interactive prototype that allows user to navigate through the input imagery and see the prediction at the location he/she points to. Similarly, set file to the location of input imagery and model.loadModel() in setup function to where the .pb model is placed.
The trained model is here, as the original file is too large to upload, it is packed in zip patches. Download all the files and unzip the street segmentation model.zip gives you the trained model (.wlnet) format
To run the model in Mathematica, see file ValidateModel.nb