/EMO-RL-UAV

The Python implementation of the proposed framework in the paper Evolutionary Multi-Objective Deep Reinforcement Learning for Autonomous UAV Navigation in Large-Scale Complex Environments

Primary LanguagePythonMIT LicenseMIT

Evolutionary Multi-Objective Deep Reinforcement Learning for Autonomous UAV Navigation in Large-Scale Complex Environments

The implementation of the proposed framework in Evolutionary Multi-Objective Deep Reinforcement Learning for Autonomous UAV Navigation in Large-Scale Complex Environments.

Notes

This program requires a long time to run, with the current settings, on a server with 8x2080GPU and Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz, 32 thread, it takes about 3.5 days to run. Each thread takes about 1GB GPU memory.

Setup

The conda environment file is env.yaml. Here we use PyTorch with CUDA 2.0.0, which might not work with the latest GPUs. Update to newer version of PyTorch using instructions from PyTorch Install Guide

crete new environment by

 conda env create -n <ENVNAME> --file env.yaml  # with pytorch 2.0.0 and cuda 11.7

or install the dependencies manually:

# REQUIRED
pytorch  # should work with any version >= 1.0.0, but only tested on 1.11, 1.13, 2.0.
gym==0.21.0  # gym has breaking changes in 0.22.0
pymoo==0.5.0  # pymoo has breaking changes in 0.6.0

# OPTIONAL
pygame  # for visualization, if not installed, you don't need to do anything with the code

REQUIRED to compile the C++ extension (to speed up execution).

# requires cmake and g++
# could be installed by:
# sudo apt install build-essential cmake
# or on Windows, install Visual Studio and CMake
# otherwise, you can just follow the procedure in the build file and compile it manually

cd gym_uav_c_functions
./build.sh
# on Windows
# build.bat

Before running, you might want to edit the configs in the main.py, which are marked with # TODO: <what is it>.

Run the experiment by:

python main.py

or run in background with:

nohup python ./main.py > out.txt 2>&1 &

Structure

  • ddpg_pytorch dir contains the implementation of DDPG with PyTorch, see the LICENSE file for more info.
  • gym_uav dir contains the implementation of the (gym) UAV environment.
  • gym_uav_c_functions dir contains the C++ extension for the UAV environment.
  • uav_problem dir contains the implementation of the (pymoo) UAV MOP.
  • main.py is the main file to run the experiment.