mattpodolak/pmaw

Length of response object can be incorrect sometimes

mattpodolak opened this issue · 1 comments

With mem_safe=True if you don't iterate through the generator fully, responses are loaded in and add to the total length, not counting how many examples have been iterated through.

Code to recreate:

comments = api.search_comments(subreddit="science", mem_safe=True, safe_exit=True, limit=2000)

len(comments) # 2000

i = 0
comment_list = []
for c in comments:
    comment_list.append(c)
    i += 1
    if i > 500:
        break
len(comments) # 3000 at this point
len(comment_list) # 501

comment_list_2 = []
for c in comments:
    comment_list_2.append(c)

len(comment_list_2) # 1499
len(comments) # 2000

Fixed in v1.0.5