Application of DQN algorithm in RL model to queue management. Data used in this project comes from COVID-2019 dataset
RL models provide a great insight to optimization. We can try to determine "optimal" strategy, by which means agent can maximize or minimize given function. By constructing an enviroment (state-space) and determining possible actions (action-space) we can create sophisticated constraints to a given task, that can serve a great representation of a field's real-world specifics. Thus we can find a balance between abstract and real, which gives us opportunity to create ready-to-use agents to solve optimization problems.
Fully working cutsom queue model fully compatible with OpenAI Gym API, and contains all necessary steps to run it out-of-the-box in Google colab.
Queue consists of patients, which are represented by vectors, where:
- time that patient spent wainting in a queue, discrete;
- severity of the given patient which a pre-dertimned and implicitly dependent of time
- various features from dataset, binary.
Thus, in each given time state-space (queue) is
Action space is discrete and described like
where:
- remove last patient,
in our case
- remove patient with lognest time, thus
- remove patient with maximal severity, thus
- remove patient with maximal linear combination of weights and binary features, thus
where:
and w - vector of feature weights
Our goal is to find a strategy (sequence of actions in each game) that can minimize
Results looks promising, but to conclude we need much more computational power and time due to enormous number of all possible state-spaces and quite unbalanced dataset.
Thanks for the idea of DQN managed queues to Yasar Sinan Nasir and Dongning Guo, Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks \ 2018, arXiv:1808.00490