international craigslist URL

Question

international craigslist URL

jonpolak opened this issue 8 years ago · 5 comments

for cities outside the US craigslist redirects away from the .org domain. e.g. vancouver.craigslist.ca - any way to configure this?

Answer 1 · 2016-08-31T00:08:28.000Z

@jonpolak I was able to get it to work outside of the US. Without knowing what you have as your arguments, can't really see where you're going wrong.

Answer 2 · 2016-08-31T16:30:01.000Z

what were the settings you used in your test? @WIRUT

Answer 3 · 2016-08-31T16:36:39.000Z

@WIRUT are you using digitalocean as your cloud provider? I find craigslist super slow when I use my own vpn server hosted at digitalocean. Maybe they are limiting digitalocean connections since lots of scrapers host their bots there?

Answer 4 · 2016-08-31T17:41:04.000Z

@VikParuchuri CONFIRMED craigslist blocks EC2 and DigitalOcean IPs

Answer 5 · 2016-09-01T09:24:06.000Z

@jonpolak I was able to test it with Calgary as well as using Amazon's EC2 services. I didn't bother using digital ocean. I'm not sure which settings you want. I did set SITE = "Calgary", and I think I bypassed the area and just hardcode the AREA in for this test instance.

Amazon EC2 was nothing different either.