It's always painful when asked at work to download hundreds of Pdfs from a website. Simply clicking "Save As" button for hundreds of time was what I did. As a bridge engineer, it's almost a route for me to download hundreds of past project pdf plans from Department of Transportation website. This repetitive work can easily finished by web crawler and that's the reason why this Get Pdfs Crawler is built.
- Download "get_pdfs.py" file into your computer
- Run "get_pdfs.py" file with python compiler
- This program will create a folder "donwload" in the directory you put the "get_pdfs.py" file
- As a example (and that's where I used ) I used Illinois DOT website here and download Culvert TSL sample plans from the website (113 files in total)
- You can assign your url link to the "base" variable in "get_pdfs.py" file and this program will download pdfs for you