To understand how protein folding problem in bidimensional hydrophobic-polar work, I provide a simple explanation here.
The main contribution of this bachelor thesis is conducting an experiment in observing the behaviour of LogDQN (Seijen and Vatemi, 2020) learning agent in discrete state space. The task is the mathematical representation of protein folding problem, bidimensional hydrophobic-polar model. In short, given a sequence of H and P symbols with length N, the objective is to find the best permutation of protein stuctures that has the lowest free energy value. The research journal, however, not published for public property.
The work from (Seijen and Vatemi, 2020) shows a potential of lower discount factor to perform better in specific task scenario than the higher one (See section 3. Analysis of Discount Factor Effects). Therefore, the main motivation in this research is to assume in longer protein sequence that result in exponential growth of the state space, there exist a specific scenario that lower discount factor will performs better in finding the lowest enegy value structure. To establish this assumtion, we leverage LogDQN, an RL algorithm that enable lower discount factor to work by performing logarithmic mapping function to the action-value function Q-value.
Below I show a screenshot of figure 1. in section 3. on (Seijen and Vatemi, 2020) paper.
The front-end code is in guiv2.py. It include all the features to either train (Figure 1.), configure the hyper-parameter (Figure 2.), simulate (Figure 3.) the folding process and evaluate the result of specific agent (Figure 4.). However, these features are not optimized well enough which occasionally cause a slow running experiences, especially when training/loading model. Therefore, it is highly recommend to perform any of these functionality directly from the back-end code.
Figure 1. Main Menu
Figure 2. Hyper-parameter Setting
Figure 3. Simulate Folding Process
Figure 4. Evaluate
The result of this experiment shows that LogDQN agent has similar performance to DQN agent with each was given the best hyper-parameter setting. Figure 5 shows the folding result of LogDQN agent following the optimial policy with 0.1 discount factor. In addition, as what have been proposed by (Seijen et al., 2019), the logarithmic mapping operation to the Q-value able to achieve decent performance with lower discount factor rather than regular Q-value from DQN agent (Figure 6.)
Figure 5. Folding Result Before and After Training LogDQN Agent with Discount 0.1
Figure 6. Learning Curve Comparison Between LogDQN and DQN with Discount 0.1 and 0.3 on Toy Dataset
- Alvin Setiadi
- Edwan Santoso
- Kelvin Chayadi
The idea of this research was initiate by Alvin Setiadi (Me). I responsible mostly for the code development in experimenting and evaluating results, while Edwan Santoso and Kelvin Chayadi collaborated mainly in constructing thesis paper.
This project is licensed under the MIT License - see the LICENSE.md file for details