Crawling data from https://www.basketball-reference.com/draft/
DDL.sql
defined the database schema, I use MySQL dialect here but theoretically it works in most common databases.requirements.txt
defined the Python packages I use.beautifulsoup
to parse html datarequests
to send http requestspymysql
to interact with database- Life is short, I use Python : )
- They write messy code (such as uncommenting code blocks in the runtime to dynamically display data 💩). Anyway, if you find some abnormal ways to parse data from the website, thanks to their 💥
- run
python3 crawler.py
to begin the journey!