Thesis written to obtain the degree of MSc in Data Science, received honours. This work explores reinforcement learning, namely policy iteration and Q-learning, to solve the famous business problem of the Beer Distribution Game. Furthermore, some assumptions regarding the behavior of the consumer demand and fields supply are changed, in order to make the problem more applicable to the real world. It was concluded that policy iteration works and is flexible and quick enough to implement. The “whiplash effect” was observed, and a valuable business insight was found: entities towards the middle of a supply chain have the highest need for well performing data science models and data scientists.
The final document can be found here
[NOTE: while the README and the code are in English, the document is written in Spanish and the commits are a strange mix of both - depending on my caffeine level, mostly. Trying to make the code accesible while complying to external language requirements has proven a massive challenge. Thank you for being patient!]
Reinforcement learning applied to a model based on agents playing The Beer Distribution Game.
The basic structure of this repository is:
aux_documents
contains documents for creating the consumer's demand, fields' supply, and a simple file showing that the explicit solution needs reiteration on itself.code
contains the following structure:
etl
, where the necessary files for creating the PostgreSQL database can be found. This database houses the results of the different iterationsmodel
contains a main file,clean_run.py
, which calls:world.py
creates the world for agentsplayers.py
creates all the agents (retailer, wholesaler, regional warehouse, factory) and non-agents (customer and fields)policy_iteration.py
trains the agents to solve The Beer Distribution Game using policy iterationq_learning.py
trains the agents to solve The Beer Distribution Game using Q-learninginsert_experiment_into_[pi/q]_database
which prepares the data on flat tables and inserts them into the database
visualizations
which contains mostly R codes to create various plots in R based on outputs
tesis_tex
containg code that generates the final.tex
file. The subfolderfigs
contains some interesting visualizations.final_documents
where the final thesispdf
andpptx
files can be found, so that local compiling is not necessary