/UAV_Navigation_DRL_AirSim

This is a new repo used for training UAV navigation (local path planning) policy using DRL methods.

Primary LanguagePython

UAV_Navigation_DRL_AirSim

A platform for training UAV navigation policies in complex unknown environment.

  • An Open AI Gym env is created include kinematic models for both multirotor and fixed-wing UAVs.
  • Some UE4 environments are provided to train and test the navigation policy.
  • based on AirSim and SB3.

Try to train your own autonomous flight policy and even transfer it into real UAVs! Have fun ^_^!

Real world transfer

Trained policy can be deployed in the real world directly!!!

ChangeLog

  • 2024-03-28
    • Deleted the LGMD part
    • Updated README file
  • 2022-03-11
    • Add W&B support
  • 2022-03-10
    • Remove gym_airsim_multirotor submodule
    • Add gym_env as envrionment, include MultirotorSimple, Multirotor and FixedwingSimple dynamics
    • Add train with plot
    • Add SimpleAvoid UE4 environment

Requirements

  • Python 3.8
  • AirSim v1.6.0
  • pytorch 1.10.1 with gpu
  • gym-0.21.0
  • Pyqt5 5.15.6
  • keyboard 0.13.5

Submodules

Install CUDA and PyTorch (Win10)

  • Download CUDA11.6
  • pip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio===0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
  • You can use tools/test/torch_gpu_cpu_test.py to test your PyTorch and CUDA.

Usage

  1. Clone this repo and the submodules

    1. git clone https://github.com/heleidsn/UAV_Navigation_DRL_AirSim.git --recursive
  2. Install gym_env

    1. cd gym_env
    2. pip install msgpack msgpack-rpc-python
    3. pip install -e .
  3. Install customized stable-baselines3

    1. cd stable-baselines3
    2. pip install -e .
    3. If you find the following error, please find the solution here
  4. Download a AirSim environment, such as SimpleAvoid.

  5. If you want to train in other environments please download AirSim released environments from here and run it.

    • Note the current version is v1.6.0 for Windows, maybe it will be updated to higher version someday.
    • If you get a directX runtime problem, please download and install it here.
    • You can use Alt+Enter to exit the full-screen mode for the first time.
    • You should copy the settings.json at the bottom of README to your /Document/AirSim/settings.json.
  6. Install other python packages

    pip install wandb pyqtgraph seaborn keyboard tensorboard tqdm
  7. Start training

    1. cd UAV_Navigation_DRL_AirSim
    2. python scripts/start_train_with_plot.py
    3. If you find it's quite slow to get data, please set ClockSpeed in your path to Documents\Airsim\settings.json over than 1 (such as 10) to speed up the training process.
    4. You log and trained model will be saved to the log folder.
  8. Evaluation

    1. cd UAV_Navigation_DRL_AirSim
    2. python scripts/start_evaluate_with_plot.py

Configs

This repo using config file to control training conditions.

Now we provide 3 training envrionment and 3 dynamics.

env_name

  • SimpleAvoid

    • This is a custom UE4 environment used for simple obstacle avoidance test. You can download it from google drive.

  • City_400_400

    • A custom UE4 environment used for fixedwing obstacle avoidance test. You can also get it from google drive

  • Random obstacles

    • Some envs with random obstacles. Contributed by Chris-cch. You can download here.

  • Other Airsim build in envrionment (AirSimNH and CityEnviron):

dynamic_name

  • SimpleMultirotor
  • Multirotor
  • SimpleFixedwing

GUI for training and evaluation

img

Wandb support

Wandb is a central dashboard to keep track of your hyperparameters, system metrics. You can find examples here.

Note: If you use wandb, please run python as administators.

Results

Training result using TD3 with no_cnn policyimg

Benchmark

2D depth navigation Benchmark for 3 different algorithms and 5 different policies in SimpleAvoid environment:

drawing

drawing

drawing

Settings

Note:

To speed up image collection, you can set ViewMode to NoDisplay.

For multirotor with simple_flight controller, please set SimMode to Multirotor. Or you can use ComputerVision mode to train without dynamics. You can also set ClockSpeed over than 1 to speed up simulation (Only useful in Multirotor mode).

Also, it's better to put your environment files in your SSD rather than HDD.

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "ViewMode": "",
  "ClockSpeed": 1,
  "SubWindows": [
    {"WindowID": 0, "CameraID": 0, "ImageType": 0, "Visible": true},
    {"WindowID": 1, "CameraID": 0, "ImageType": 3, "Visible": false},
    {"WindowID": 2, "CameraID": 0, "ImageType": 3, "Visible": true}
    ],
  "CameraDefaults": {
    "CaptureSettings": [
      {
        "ImageType": 3,
        "Width": 100,
        "Height": 80,
        "FOV_Degrees": 90,
        "AutoExposureSpeed": 100,
        "AutoExposureBias": 0,
        "AutoExposureMaxBrightness": 0.64,
        "AutoExposureMinBrightness": 0.03,
        "MotionBlurAmount": 0,
        "TargetGamma": 1.0,
        "ProjectionMode": "",
        "OrthoWidth": 5.12
      },
      {
        "ImageType": 0,
        "Width": 256,
        "Height": 144,
        "FOV_Degrees": 90,
        "AutoExposureSpeed": 100,
        "AutoExposureBias": 0,
        "AutoExposureMaxBrightness": 0.64,
        "AutoExposureMinBrightness": 0.03,
        "MotionBlurAmount": 0,
        "TargetGamma": 1.0,
        "ProjectionMode": "",
        "OrthoWidth": 5.12
      }
    ]
  }
}