CRutkowski/Kijiji-Scraper

Stopped working?

Closed this issue · 4 comments

grabosky commented

Hi there,

The script seems to stop working on October 21st around 14:00 hrs, could you confirm that?

I have 2 copies of the script running on 2 different servers (and using 2 different e-mails) and now I'm getting "Found 0 new ads" every time I run the script.

I ereased ads.json to clear things up a bit, but it is not getting repopulated with new ads. The script still shows "Found 0 new ads", and new json file contains "{ }"

Is it due to some Kijiji website changes?

Best,
Adam

ZaM-Rai commented

You'll have to change the following commented line to the uncommented line in kijiji_scraper.py
#kijiji_ads = soup.find_all("div", {"class": "search-item regular-ad"})
kijiji_ads = soup.find_all("div", {"class": "search-item"})

I think the third party ad has also changed though though, didn't check how to remove that one.

  • 👍1
CRutkowski commented

Thanks for the heads up, I will check this out tomorrow.
Seems Kijiji may have changed their site a bit and altered the div class names.

grabosky commented

Thanks guys, it works!

For third party ad, would that be?
third_party_ads = soup.find_all("div", {"class": "adD"})

CRutkowski commented

I updated the div class names in #17 so the scraper should be functional again.
Let me know if it still doesn't work for you.

  • 👍1