The video below shows a Reinforcement Learning agent trying to find increasingly faster paths from (0,0) to (3,-10) via only the influence of gravity. The orange curve is the optimal solution (the Brachistochrone), which the agent has no idea about. The best path found by the agent is the blue curve.
Screen.Recording.2023-10-13.at.18.38.46.mov
Brachistochrone.py just tries to guess faster paths from point A to point B under the influence of only gravity, and updates the fastest path via a matplotlib animation.
RL_Brachistrochrone.py implements a reinforcement learning environment with a neural network that tries to learn the best path under the the influence of only gravity, and also symbolic regression to produce candidate equations based on the coordinates of the best path found by the reinforcement learning agent. The paths are also shown in real-time via a matplotlib animation. An example output is shown in RL_Brachistrochrone.png
- Create and start virtual environment
python3 -m venv myvenv
source myvenv/bin/activate
- Install requirements.txt
cat requirements.txt | xargs -n 1 pip3 install
After installing requirements.txt, make the following file modifications (substituting your environment name with myvenv and your python version with python3.9):
- Change
if hasattr(actor.output, '__len__') and len(actor.output) > 1:toif hasattr(actor.output, '__shape__') and len(actor.output) > 2: - Change
if hasattr(critic.output, '__len__') and len(critic.output) > 1:toif hasattr(critic.output, '__shape__') and len(critic.output) > 2: - Comment out
if self.uses_learning_phase:,critic_inputs += [K.learning_phase()],inputs += [self.training]
- Comment out the following:
if len(data_shape) != len(shape):
raise ValueError(
"Error when checking "
+ exception_prefix
+ ": expected "
+ names[i]
+ " to have "
+ str(len(shape))
+ " dimensions, but got array with shape "
+ str(data_shape)
)Then you can run the program:
python3 RL_Brachistochrone.py