This repository features R code designed to teach students web scraping in R
ATTENTION: Web scraping is not conducted in an unregulated, norm-free space. Please consider the following:
- Consult the robots.txt file of each website whether you are allowed to scrape data from a web directory!
- Be mindful to the service providers. Implement time lags between your requests to the web server!
- Consider scheduling your requests outside peak times (e.g. during the night)
The following packages are used: rvest, httr, tidyverse, data.table