Python Reinforcement Learning for FTC
This project was originally created for a senior year capstone project. My inititial goal was to create a system that would create the foundation for a fully autonomous teleop program. Now that the school project has ended, I plan on creating a little bit of documentation in case other people would like to contribute to this project.
Currently, this project contains a large portion of code unrelated to the project and test and legacy code. Additionally, the sctructure of the code could be much cleaner. If other individuals are interesting in working on the project, this will become an a major area to work on.
Right now, some results have been scucesfully transfered to real life. There are a lot of different approaches I would like to try, which I will be working on while I still have time.
This project is also a reinforcement learning framework using tensorflow-keras where I implement new algorithms as I learn them. Currently, the project is just compatible with the custom mayavi environment, however there will soon be suppport for OpenAI gym environments. Currently, gym environments with single discrete action spaces that are epoch-based should work, however this has not been tested.
The framework supports training arugument loading from files and arguments for the mayavi environment from files. Addtionally, you can asynchronously train multiple agents at once to test differnet hyperparameters and algorithms.
Here is list of implemented algorithms:
- Deep Q Networks (DQN) with state-action, state, and multi-state (frame stacking) model input
- Improvements such as Dueling and Double (target) networks
- Deep Recurrent Q Networks (DQRN)
- Combining algorithms, such as Deep Double Dueling Recurrent Q Networks (DDDQRN)
coming soon, main project code is contained within: https://github.com/daniellawson9999/Capstone_Python/tree/master/Robot/Rotation
For now, here are some slides and videos about the project. I can attach scripts soon:
- First Presentation
- Most Recent Presentation
- Final Presentation
- Final Presentation Script
- Google Drive folder of real life and simulated photos and videos
If you would like to know more about deep q learning, I would recommend the Playing Atari with Deep Reinforcement Learning which presented this approach.
Currently, it can be difficult to properly setup your dependencies. I will make a recommended setup guide and will eventually use a service such as docker.
https://github.com/daniellawson9999/RL-Robot-Code
Support for most gym environments Improved training metrics (reward and performance graphs) and additional tensorboard metrics Testing in linux in headless environments
Algorithms: Adding prioritized experience replay (with importance sampling) Implementing policy based algorithms, starting from: Monte Carlo Policy Gradients with REINFORCE Actor Critic Advantage Actor Critic (A2C)
Discord: Daniel_Lawson9999#7898 (preferred) Email: daniellawson9999@gmail.com