This reopistory contains the code related to training a DNN model to predict depth from camera and feature motion. The theoretical details are found in the thesis of James Adams titled A Series of Improved and Novel Methods in Computer Vision Estimation (link coming soon).
The model is trained on AirSim recording files. Some changes to the AirSim plugin are needed to record all of the desired information, specifically the angular rates of the aircraft. In light of this, environments in which AirSim is prepackaged cannot be used as the plugin cannot be modified. To use an environment to train the model, a different Unreal environment must be used and the AirSim plugin must then be added. We used this park environment, which was available for free. It may be possible for the camera, depth, and state information to all be obtained by a python script that simultaneously flies waypoints for the virtual drone, in which case the prepackaged AirSim environments may be used. We believe there were issues obtaining the depth images from the virtual drone by python script, and this is why the recording was used.
Once the recording has been obtained, this code can be setup in ROS. ROS can then be used to run airsim_recording_tracker.launch, which utilizes a feature tracker to track features in the scene. It does this judicously, only registering features after they have been tracked for a number of frames. This ensures only high quality features are reported. It then calibrates the features according to the camera parameters in the yaml files and recordes their location and velocity. For training, parse_airsim_recording.py is then used to create individual lines for use in the training. It takes the features recorded and pairs it with the camera motion from the corresponding camera frame. One line is produced for every desired depth pixel in the image for every camera frame. The resulting file can be very large for a large recording. Some slicing may be required.