mattpodolak/pmaw

Successful Rate 0% when batch loading comments by ids

dukesun99 opened this issue · 1 comments

I found a strange bug, basically when calling search comments, large max_ids_per_request will result in 0% successful rate and forever retry. Maybe the result is too long?

To reproduce:

comments = api.search_submission_comment_ids(ids="haucpf")    
comment_list1 = [comment for comment in comments]    
comments = api.search_comments(ids=comment_list1, max_ids_per_request=1000)    
comment_list_full = [comment for comment in comments]    

The above code works if we change to max_ids_per_request=500.

Hi @dukesun99 it looks like Pushshift has started returning 520s for requests with 1000 ids, I'll include an update in the upcoming v2 release to set the maximum value to 500, thanks for reporting this issue