campaignmonitor/createsend-ruby

1000 limit for API responses

Closed this issue · 9 comments

robyedlin commented

I'm running API calls and only getting the first 1000 results returned. Is there a way to get all results, or at least pass in a page number?

# List has 1300 active subscribers
list = CreateSend::List.new CAMPAIGN_MONITOR_AUTH, intro_list_id
# The following returns:
#1000
puts list.active['Results'].count
robyedlin commented

This is what I came up with as a workaround. It seems like we have to grab the number of pages and loop through the pages one at a time. It's a little cumbersome:

list = CreateSend::List.new CAMPAIGN_MONITOR_AUTH, intro_list_id
number_of_pages = list.active['NumberOfPages']
i = 1
email_addresses = []
while i < (number_of_pages.to_i + 1) do
    # The active method on list seems to take in params, the second one being the page number
    list.active("",i)['Results'].each do |result|
    email_addresses << result['EmailAddress']
    end
    i += 1
end
# The following returns 1300, which is the correct number for my list
puts email_addresses.count
robyedlin commented

Thanks @mjwills. Is the way I'm doing it the recommended way? Is the API going to get grumpy if I make 1,000 calls in this loop (1 million emails)

mjwills commented

It looks fine to me. You could experiment with increasing page_size - but honestly if what you have is working then I would just do what you are doing. 1000 API calls doesn't seem crazy, unless you are doing the whole list (repeatedly) every second. :)

robyedlin commented

It might be once a second, because I'll be scraping a list of 1 million, and processing them in batches of 1000.

mjwills commented

Gotcha.

And how often will you do that? Once-off? Once a day? Once a month? Once a year?

robyedlin commented

It's a daily sync.

mjwills commented

Your code is perfect then.

robyedlin commented

Ok, thanks!