python-web-scraping

Hands-on workshop material on Web scraping using Python 🛠️⚙️

To build and run site locally

Execute the following commands

git clone https://github.com/MonashDataFluency/python-web-scraping.git
cd python-web-scraping
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
mkdocs serve

To update the website

After modifying any notebook content, please run the ./compile.sh script to update the website or alternatively :

Run jupyter nbconvert --output-dir='markdowns/' --to markdown notebooks/*.ipynb from the root directory to generate the markdown files from jupyter notebooks, and
Run mkdocs build to build the website again.

Troubleshooting

In case wptools throws an error during installation please install the following dependencies first :

sudo apt install libcurl4-openssl-dev libssl-dev

and then proceed to install wptools though pip install -r requirements.txt as above.

License

This workshop material is made available under a Creative Commons Attribution license (CC-BY 4.0)

Code is made available under the MIT License.