pagination doesn't respect sleep time
Closed this issue · 7 comments
for some reason.
It should.
Ah so I didn't realize I was posting in the wrong area. We are coming across this issue as well. From what I have been reading in the code, it looks like the sleep time will only work if stashing is disabled. That is the only time I can find resp_and_cache[:from_resource] being set to true as required here. At least that is my take from what I am reading in the get method. Would it make sense to write the sleep into the download_from_resource! method?
Lol no worries, it's whatever.
Ah figured it out
fix tk shorlty
bccd019#diff-9347901060f5e801766116fbb544a20dL80
The boolean here was flipped; literally telling it not to sleep when it should have told it to sleep. Silly Jeremy.
Hey, I am running an array of urls through the scraper with no pagination. Sleep time is being ignored. I am working on subclassing some stuff to work around this, but I thought you should know.
Hmm. Can you make a gist or something so I can test it?
Also, can you make sure you're pulling the latest master from github? I didn't push an updated gem yet.
I pulled the master after you updated. Will send you a gist shortly.
On Thu, Dec 19, 2013 at 11:38 AM, Jeremy B. Merrill <
notifications@github.com> wrote:
Hmm. Can you make a gist or something so I can test it?
Also, can you make sure you're pulling the latest master from github? I
didn't push an updated gem yet.—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-30943796
.