Neural-Fly enables rapid learning for agile flight in strong winds

Michael O'Connell¹, Guanya Shi¹, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung²

This data and code is provided as part of the Science Robotics research article "Neural-Fly enables rapid learning for agile flight in strong winds", published on May 4th, 2022 here.

Training and validation script

Please run training-and-validation.ipynb, which demonstrates the Domain Adversarially Invariant Meta Learning (DAIML) algorithm. DAIML is the offline learning process for Neural-Fly. This script trains a wind-invariant representation of the aerodynamic effects on a quadrotor. After training the model, some simple statistics and plots are generated which show the model performance fitting to the training and testing data.

Filenaming scheme

Filenames are structured as

<VEHICLE>_<TRAJECTORY>_<METHOD>_<CONDITION>.csv

For specific details please refer to the article.

Field	Description
`<VEHICLE>`	`custom`: drone built using consumer off-the-shelf components with PX4 flight controller `intel`: Intel-Aero RTF Drone, used for data collection for Neural-Fly-Transfer controller
`<TRAJECTORY>`	`random3`: randomized trajectory created by randomly sampling 2 waypoints and generating a smooth spline from the current position through both waypoints; continuous derivatives through snap `random2`: similar to `random3` except only one random waypoint is generated `figure8`: a lemniscate trajectory given by `(x(t),y(t),z(t)) = (1.25 * sin(t), 0, 0.75 * sin(2 * t)`
`<METHOD>`	`'baseline'`: nonlinear baseline control `'indi'`: incremental nonlinear dynamics inversion control `'L1'`: L1 adaptive control `'NF-C'`: Neural-Fly-Constant, our adaptive controller without any learning `'NF-T'`: Neural-Fly-Transfer, our learning based adaptive control with the ML model trained on data from the Intel-Aero drone `'NF'`: Neural-Fly, our learning based adaptive control with the ML model trained on data collected with the custom drone ,
`<CONDITION>`	wind condition for experiments, where the number corresponds to the fan array duty cycle and converts to `nowind` = 0 m/s `10wind` = 1.3 m/s `20wind` = 2.5m/s `30wind` = 3.7 m/s `35wind` = 4.2 m/s `40wind` = 4.9 m/s `50wind` = 6.1 m/s `70wind` = 8.5 m/s `70p20sint` = 8.5+2.4sin(t) m/s `100wind` = 12.1 m/s

Experiment data

Additionally, the data from the experiment results present in the paper is provided. To load the data, run the following in python

import utils
Data = utils.load_data(folder='data/experiment')

This will load all of the experiment data as a list of dictionaries. The ith experiment, field field, at the jth timestep, can be accessed with Data[i][field][j]. Most fields are ndarrays except the metadata fields, pulled from the filename. Available fields are given in the following table.

`field`	description
`'t'`	time in seconds
`'p'`	position vector in meters
`'p_d'`	desired position vector in meters
`'v'`	velocity vector in m/s
`'v_d'`	desired velocity in m/s
`'q'`	attitude represented as a unit quaternion
`'R'`	rotation matrix (body frame to world frame)
`'w'`	(this is a misnomer - should be $\omega$) angular velocity in rad/s
`'T_sp'`	thrust setpoint sent to the flight controller
`'q_sp'`	attitude command sent to the flight controller
`'hover_throttle'`	throttle at hover computed as a best fit function of the battery voltage
'`fa'`	aerodynamic residual force computed using numerical differentiation of `v` and `T_sp`, `q`, and `hover_throttle`
`'pwm'`	motor speed commands from the flight controller
`'vehicle'`	`<VEHICLE>` field from filename
`'trajectory'`	`<TRAJECTORY>` field from filename
`'method'`	`<METHOD>` field from filename
`'condition'`	`<CONDITION>` field from filename

Videos

Check out our overview video:

60 second overview video created by Caltech's Office of Strategic Communications:

Citation

The data and code here are for personal and educational use only and provided without warranty; written permission from the authors is required for further use. Please cite our work as follows.

@article{ doi:10.1126/scirobotics.abm6597, author = {Michael O’Connell and Guanya Shi and Xichen Shi and Kamyar Azizzadenesheli and Anima Anandkumar and Yisong Yue and Soon-Jo Chung }, title = {Neural-Fly enables rapid learning for agile flight in strong winds}, journal = {Science Robotics}, volume = {7}, number = {66}, pages = {eabm6597}, year = {2022}, doi = {10.1126/scirobotics.abm6597}, URL = {https://www.science.org/doi/abs/10.1126/scirobotics.abm6597}}

equal contribution and alphabetical order ↩ ↩²
corresponding author - sjchung@caltech.edu ↩