TristanBidouard/UnityPySocket

Robot arm with machine learning on Unity

Deep reinforcement learning for a robot arm via a Unity-Python interface

The main goal of this project is **to allow a robot arm to learn by itself how to press a red button**.

Environment

An Unity environment was created to simulate the robot and his envirnement.

• The robot is in the center
• The red cube represents the button
• The green sphere represents the cube potential position
• The blue area represents the cube position limit
• The output camera is displayed at the bottom right
• A start button is implemeted to launch the simulation

The shell displays some information about the program proceedings.

The results are represented as graphs.

A server makes the link between ou python script and our .exe simulation. It then allows to communicate the robot.

Algorithm

The chosen solution is to use a deep reinforcement learning algorithm.

For information, some good courses about this method :

• https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0
• https://www.intelnervana.com/demystifying-deep-reinforcement-learning/
• http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
• http://cs231n.github.io/

The algorithm use :

• A convolutional neural network (CNN) based on DeepMind to find the action from the robot state
• A policy gradient approach to update the CNN (loss = -log(π)*A)

Getting started

Installation

Prerequisites

• OS : Windows (Linux in progress)

Dependencies

• TensorFlow
• Numpy
• PIL
• Matplotlib

Clone GitLab environment

In your personnal folder, clone the GitLab environement:
git clone https://gitlab.com/RoboAcademy/UnityPySocket

WARNING ! Before continue, you have to unzip the Unity build in `(your folder)\UnityPySocket\Unity\Build\Build`

Usage

Launch the script

From (your folder)\UnityPySocket\Python\Python, launch the script:
python main.py

Several keystrokes will be requested from the user.

1. Open Unity environment (y/n) : Open robot simulation
2. Name of the built executable: : Inform the name of the .exe file
3. Use manual position (y/n) : 2 types of script (manual or learning)

Restore session (y/ n) : Relaunch an existing simulation (if learning script is chosen)

5. Click on the start button of the robot environment

Here we go ! The simulation is launched !

Authors

• Jacob THORN
• Tristan BIDOUARD
  
  With the help of :
  
• Santiago QUINTANA-AMATE
• Pablo BERMELL-GARCIA
• Kiran KRISHNAMURTHY
  

From _AIRBUS GROUP UK_