SHIP AI

Ship AI is a reinforcement learning module based on the OpenAI Gym and Keras-RL projects. This module presents a simulation environment for navigating a ship in a channel (Restricted Waters). The purpose of the navigation task is to keep the ship in the midline of the channel. With ShipAI you can:

Design Reinforced Learning (RL) algorithms to control ship maneuver movements.
Visualize the Ship movement Behavior over a channel.
Train, test and validate your RL agent.

States

The main purpose of the navigation task was defined as follows:

Use the rudder and propulsion controls to perform a defined linear navigation path along a channel

The states chosen for the application of RL in the task of the ship were as follows:

Where is the distance from the center of mass of the ship to the guideline; $\theta$ is the angle between the longitudinal axis of the ship and the guideline; is the horizontal speed of the ship in its center of mass (in the direction of the guideline; is the vertical speed of the ship in its center of mass (perpendicular to the guideline); is the angular velocity of the ship.

Reward

The reward function is defined as:

Where:

Actions

The action are the input parameters for controlling the ship maneuver movement. The forces that make the ship controllable are the rudder and propulsion forces. They have the vector form of , where is the dimensionless rudder command and the dimensionless propulsion command, such that and . These parameters have a direct proportional relation with the rudder angle and the propulsion

How to install ShipAI?

Make sure you have all necessary modules installed:

pip install TensorFlow
pip install Keras
pip install gym
pip install keras-rl
pip install Shapely
pip install numpy
pip install scipy

Clone or download the repository:

git clone https://github.com/jmpf2018/ShipAI
cd ShipAI

Run DQN or DDPG RL algorithms:

python dqn_keras_rl.py or
python ddpg_keras_rl.py

4.After that RL will start to run:

5.Finally in the end of the process the vizualization screen you start:

Final Considerations

You can change the ship dynamics and adapt for your needs in: simulator.py
You can change the reward and state design in: ship_env.py
You can change the network hyperparameter in ddpg.py or dqn.py
You can evaluate the learing process and the states evolution in evaluate_learning.py

Paper

To read our paper check this link.

How to cite us?

@misc{shipAI,

author = {Jonathas Pereira and Rodrigo Pereira Abou Rejaili},

title = {ShipAI},

year = {2018},

publisher = {GitHub},

journal = {GitHub repository},

howpublished = {\url{https://github.com/jmpf2018/ShipAI}},

}