The goal is to use deep reinforcement learning algorithms: Deep Deterministic Policy Gradient (DDPG) to control mobile robot(turtlebot) to avoid obstacles while trying to arrive a target.
Goal: Let robot(turtlebot) navigate to the target(enter green circle)
Demo video (Speed up ten times )
Input(State):
-
Laser finding (10 Dimensions)
-
Past action (Linear velocity & Angular velocity) (2 Dimensions)
-
Target position in robot frame (2 Dimensions) a. Relative distance b. Relative angle (Polar coordinates)
-
Robot yaw angular (1 Dimensions)
-
The degrees to face the target i.e.|Yaw - Relative angle| (1 Dimensions)
Total: 16 Dimensions
Normalize input(state):
-
Laser finding / Maximum laser finding range
-
Past action (Orignal)
-
Target position in robot frame
a. Relative distance / Diagonal length in the map
b. Relative angle / 360
-
Robot yaw angular / 360
-
The degrees to face the target / 180
Output(Action):
Linear velocity (00.25 m/s) (1 Dimensions)
Angular velocity (-0.50.5 rad/s) (1 Dimensions)
Reward: Arrive the target: +120 Hit the wall: -100 Else: 500*(Past relative distance - current relative distance)
Algorithm: DDPG (Actor with batch normlization Critic without batch normlization)
Training env: gazebo
Python3
Tensorflow
pip3 install tensorflow-gpu
ROS Kinetic
http://wiki.ros.org/kinetic/Installation/Ubuntu
Gazebo7 (When you install ros kinetic it also install gazebo7)
http://gazebosim.org/tutorials?cat=install&tut=install_ubuntu&ver=7.0
cd
mkdir catkin_ws && mkdir catkin_ws/src
cd catkin_ws/src
git clone https://github.com/m5823779/MotionPlannerUsingDDPG.git project
git clone https://github.com/m5823779/turtlebot3
git clone https://github.com/m5823779/turtlebot3_msgs
git clone https://github.com/m5823779/turtlebot3_simulations
cd ..
catkin_make
And add following line in ~/.bashrc
export TURTLEBOT3_MODEL=burger
source /home/"Enter your user name"/catkin_ws/devel/setup.bash
Then enter following command in terminal
source ~/.bashrc
Demo: First run:
roslaunch turtlebot3_gazebo turtlebot3_stage_1.launch
In another terminal run:
roslaunch project ddpg_stage_1.launch
Train: If you want to retrain yourself change the setting
is_training = True # In: project/src/ddpg_stage_1.py
Idea:
https://arxiv.org/pdf/1703.00420.pdf
Network structure:
https://github.com/floodsung/DDPG
Ros workspace:
https://github.com/ROBOTIS-GIT/turtlebot3
https://github.com/ROBOTIS-GIT/turtlebot3_msgs