Use list instead of collections.deque for replay buffer
emailweixu opened this issue · 0 comments
The current replay buffer at python/social_bot/util/replay_buffer.py uses collections.deque for storage. However, deque can have performance issue when the size is very large. As shown in deque implementation of deque_item https://github.com/python/cpython/blob/3.5/Modules/_collectionsmodule.c#L1123
To access an item, it needs to traverse through all the blocks of size 64, which can be time-consuming when size is large. As I run examples/train_simple_navigation.py, the fps decreases from 45 to 35 and stops decreases after 500,000 steps. It's coincident with the size of replay buffer. So I suspect the longer random accessing time of deque is cause of this problem.
We should use a cycled list to replace deque so that it will not get slower as size gets large.