🔥 Since the country went into lockdown, people started hamstering and our beloved collect&go system was constantly overloaded. It's a webshop where you just order your groceries & pick a time to pick them up at the store. But since the Covid.19 outbreak, everyone got into this service which made us go to the stores ourselves.. 🎪
After a couple of weeks, Colruyt released a table containing all store locations and if they have open spots for pickup or not. This seemed like the perfect use case for my first webscraping application!
I started by finding one of the easiest webscraping tools: puppeteer! After a couple of minutes I got my first scrape and I was hooked! I quit the tutorial and started implementing it for my case.
My first page immidately got labeled as a bot, but I came prepared, I knew this would happen. So I auto-filled some data the website needed to verify that I'm human 😈
But the next page suddenly became the best detective ever and blocked my browser..
To be continued...
(100% JavaScript)
- HTML / CSS / Vanilla JS (no framework)
- express
- Puppeteer
- TypeORM
- mySQL
collect-and-go-scraper>node ./server/index.js
collect-and-go-scraper>start ./client/index.html