datagovuk-scraper

Scrap public data from data.gov.uk without an API KEY

Get started 🎉

Clone the project

git clone https://github.com/av1m/datagovuk-scraper
cd datagovuk-scraper

Install dependencies
```
pip install -r requirements.txt
```
Run the project 🚀
You can run the project in different ways.
- To get started quickly, you can use the notebook.
- Plus, you can use the command line
- Or, directly in your code by importing the datagovuk module.

CLI Usage 📖

💡 You can directly install dependencies instead of clone the repository :
pip install git+https://github.com/av1m/datagovuk-scraper.git

To run the project from the command line, use the command datagovuk.

Here is the list of available commands:

datagovuk --help
usage: __main__.py [-h] --query QUERY --number-record NUMBER_RECORD [--output {csv,ods,html,pdf,xls,zip}] [-d] [-v]

Scrap public data from data.gov.uk

options:
  -h, --help              show this help message and exit
  --query, -q             Search query (required)
  
  --number-record, -n     Number of records to fetch, must be a valid number (required)
  --output, -o            Output file (must be csv,ods,html,pdf,xls,zip). Default is csv
  -d, --debug             Put the logger in debug mode (default: Warning)
  -v, --verbose           Put the logger in info mode (default: Warning)

And, here an example of some examples:

Get the first 10 records for the query "house" and with only CSV file as output (debug mode activated):
```
datagovuk -q house -n 10 -o csv -d
```