/webscrap-sample

scraping millions of records from a public government website

Primary LanguagePython

Web Scraping Sample

A script and tools to scrape a whole country's population data from a web service and put them in a database.
Threaded to make the ~19 million records faster to download.
Separate script to verify downloaded records and a small Flask app to view the data.