Meta-Reinforcement Learning for Spacecraft Proximity Operations Guidance and Control in Cislunar Space
In order to tackle the challenges of the future space exploration, new lightweight and model-free guidance algorithms are needed to make spacecrafts autonomous. Indeed, in the last few decades autonomous spacecraft guidance has become an active research topic and certainly in the next years this technology will be needed to ensure proximity operation capabilities in the cislunar space. For instance, NASA’s Artemis program plans to establish a lunar Gateway and this type of autonomous manoeuvres, besides nominal rendezvous and docking (RV&D) ones, will be needed also for assembly and maintenance procedures.
In this context a Meta-Reinforcement Learning (Meta-RL) algorithm will be applied to address the real-time relative optimal guidance problem of a spacecraft in cislunar environment. Non-Keplerian orbits have a more complex dynamics and classic control theory is less flexible and more computationally expensive with respect to Machine Learning (ML) methods. Moreover, Meta-RL is chosen for its elegant and promising ability of ‘‘learning how to learn’’ through experience.
A stochastic optimal control problem will be modelled in the Circular Restricted Three-Body Problem (CRTBP) framework as a time-discrete Markov Decision Process (MDP). Then a Deep-RL agent, composed by Long Short-Term Memory (LSTM) as Recurrent Neural Network (RNN), will be trained with a state-of-the-art actor-critic algorithm known as Proximal Policy Optimization (PPO). In addition, operational constraints and stochastic effects will be considered to assess solution safety and robustness.
This project has been created by Giovanni Fereoli in 2023. For any problem, clarification or suggestion, you can contact the author at giovanni.fereoli@mail.polimi.it.
The package is under the MIT license.