This repo demonstrates a novel process for remaining useful life (RUL) estimation using Variational Autoencoders (VAE) and scalable Reinforcement Learning (RL). The dataset used is the train_FD002.txt and can be found in the CMAPSSData directory.
The proposed process evolves in two steps:
- Split the data into three subsets i) VAE training, ii) RL agent training, iii) validation
- Train a VAE using the first subset of the training data. The decoder is then used in lieu of an actual simulator.
- Train an RL agent using the second training data subset, and the trained decoder as environment with which it interacts. The actions are resembling generated distribution samples from the VAE's encoded layer
- Evaluate the agent's performance by estimating the RUL of the systems on the third training data subset.
The files provided are:
config.py: Predetermined hyperparameters for the models used.data_prep.py: data preprocessing step; includes data normalization, actual life estimation, smoothingVAE_dense.py: VAE training process.env.py: The RL environment; inherits from the OpenAI Gym module.rllib_trainer.py: RL training process, works via the Ray RLlib module.
For agent evaluation, you need to run the RLlib-PPO-policy_eval Jupyter notebook.
Modules needed for running the experiments:
gym
ray[rllib, serve]
numpy
pandas
itertools
tensorflow
requests
scikit-learn
argparse
jupyter
matplotlib
bisect
Contact Erotokritos Skordilis for any questions/comments.