psychNerdJae/learning-rl

Reinforcement learning as used in psychological research

Learning RL

Jae's record of self-teaching reinforcement learning (RL), as it's used in psychological research. Also a record of self-teaching Python(!), so lacks any sort of the elegance of well-written Python.

Topics

1. Learning the value of a single stimulus (learning rate parameter alpha)
2. Multi-stimulus environments (Rescorla-Wagner and the prediction error delta)
3. Choosing between stimuli (exploration parameter beta)
4. Multi-step choice (temporal discounting parameter gamma)
5. Agents with "memory activation" (TD-lambda)
6. Sarsa and Q-learning
7. Model-based (MB) agents and transition structure
8. The Successor Representation (SR) and transition structure

Potential future topics

1. SR + Dyna replay + prioritized sweeping
2. SF + LSFM
3. Parameter-fitting

Inspiration / tutorials

• Bob Wilson's class on computational modeling
• Hanneke den Ouden and Jill O'Reilly's tutorial on reinforcement learning
• Ida Momennejad's GitHub repository on predictive representation