Smarter polling [long-term issue]
Opened this issue · 2 comments
I currently have around 1400 users that my installation is polling for content. It takes about 30 minutes to get through the list. Many of the accounts have not had any updates in a long time but the still get polled just as frequently as users who have recent submissions. In the long run this is going to be a big problem and will most likely result in missing out on content.
One thing that has occured to me is to create a group of users who have been inactive for X time period. The process could poll some subset of this group after grabbing the new top content week. Then go through the list of active users again like it currently does.
One option is to use the "friends" functionality (--just-friends), which only polls /r/friends for updates every 2 seconds.
It should still work...
Then again, the last-updated time for each user is available in the database... Coming up with an algorithm that checks new accounts quickly and occassionally checks older accounts would require some thought.