Reinforcement Learning applied on predictive maintenance & machine replacement usecase
Jupyter Notebook
rl-predictive-maintenance
Problem statement
At the end of each production cycle (e.g. seasonal) a candy production line must decide
whether to keep a machinery again or replace it with a new one. A machinery at cycle t has
a corresponding efficiency state $s_t \in S = {1, 2, ..., 10}$. We know the machinery’s state at the
first cycle $s_1 = 1$, and it has probability $p = 0.9$ to go to efficiency state $s_{t+1} = min{s_{t+1}, 10}$
and probability $1 − p$ to go to efficiency state $s_{t+1} = min{s_{t + 2}, 10} if not replaced by a
new one. At each efficiency state $s$, it produces $y(s) = 8 + s − 0.15s^2$ tons of candy over the
corresponding production cycle. We assume a machinery must be replaced upon completion
of the production cycle $s_t = 10$ since it becomes too unproductive. The net cost of replacing
a machine is $c = 500$ k€ and the profit contribution of candy is $m = 150$ k€ per ton.
Problem formulation
Write down the Bellman optimality equation of the value function.
Find the replacement policy that maximizes the expected long term cumulative profits using Value Iteration to solve the problem (Policy Iteration and Linear Programming methods can be considered too)
Test the sensitivity of the optimal policies to different problem parameters